Schrieffer was born in Oak Park, Illinois, on May 31, 1931. He earned his bachelor's degree from the Massachusetts Institute of Technology and his master's (1954) and doctoral (1957) degrees from the University of Illinois. While a graduate student, Schrieffer was invited by John Bardeen to work with him on the problem of superconducting theory.

Superconductivity had been discovered in 1911 by Heike Kamerlingh-Onnes. Over the next 40 years, physicists learned a great deal about the superconducting state, but had developed almost no theoretical explanation for their empirical observations. By the early 1950s, the stage was set for such a theory. F. London, W. Meissner, R. Ochsenfeld, H. Fröhlich, and D. Pines, among others, had laid out some of the fundamental propositions that would be necessary in any theory of superconductivity. Between 1955 and 1956, Bardeen, Schrieffer, and Leon Cooper developed such a theory.

The first clue to the theory, known today as the BCS theory, arose out of Cooper's discovery of paired electrons, now called Cooper pairs. Such pairs develop in a superconductor when two electrons interact with the positively charged ionic matrix of a metal in such a way that there develops a force of attraction greater than the normal electrostatic force of repulsion between the electrons. Schrieffer's task was to find a system of mathematical equations that could be used to generalize the phenomenon of Cooper pairs to all electrons in a metal.

The task seemed daunting at first. In fact, it is said that Schrieffer was so discouraged that he considered changing the topic of his doctoral research from superconductivity to ferromagnetism. He stayed with the problem, however, and eventually found a statistical procedure for dealing with 1023 Cooper pairs all at once. Schrieffer's solution was so successful that the BCS theory is now able to account for all observed properties of superconductors. Schrieffer described his research and findings in his Theory of Superconductivity, published in 1964. For his part in the development of this theory, Schrieffer shared the 1972 Nobel Prize for physics with Bardeen and Cooper.