The youngest of five children, Baldwin was born in Elizabethtown, New Jersey. His father, a carriagemaker, died when Baldwin was five, and the fortune he left to his wife was squandered by lawyers. Baldwin's mother, however, saw to it that her youngest son received an education and apprenticed him to a jeweler in Philadelphia.

Baldwin soon tired of jewelry making and in 1825 decided to open a manufacturing business along with a partner, David Mason. Baldwin demonstrated a great deal of talent in manufacturing, producing a flurry of products in succession, from bookmaking tools to textile and printing processes. Mason, unnerved by Baldwin's engineering prowess, quit the business.

In 1828 Baldwin had constructed a steam engine to run equipment for his firm and when he had the opportunity to examine a locomotive that had been imported to the United States from England, he decided to try his hand at building locomotives. This was a timely decision, for the crude locomotives being built at that time were ripe for improvement. Baldwin, with his keen engineering skills, succeeded immediately. Within two years Baldwin constructed Old Ironsides, a six-ton locomotive, for the Philadelphia and Germantown Railroad. Old Ironsides became one of the first practical locomotives in America, making daily trips between the two Pennsylvania cities. Over the next ten years, Baldwin built ten more locomotives, and, by the time of his death in 1866, his company had built 1,500. The efforts of Baldwin, who epitomized the entrepreneurial engineer of nineteenth-century America, helped lay the foundation for the railroad revolution that later swept through the United States.

The company that Baldwin founded, the Baldwin Locomotive Works, became one of the world's great industrial giants. Yet despite its experimentation with steam turbines and diesel engines after 1939, the company never was able to make a satisfactory tranisition from steam and was eventually put out of business by two newcomers, General Motors and General Electric.