Having lost both parents at a very young age, Jonas Crow lives with his aunt and uncle for several happy years. However, when they, too, pass away Jonas is sent to an orphanage where his identity becomes just a letter in the alphabet, "J." He will always remember the cold stare of authority at the orphanage. Eventually earning a scholarship to study for the ministry, Jonas develops troubling questions about the Bible and about certain Christian beliefs related to the soul and body. He realizes the ministry may not be for him.

As a young adult in 1937, Jonas makes his way through a horrendous record-breaking flood back to the small settlement in Kentucky called Port William, powerfully drawn to his happiest, early memories of a simpler life. He becomes the town's only barber and is known now by his friends as Jayber. He spends his adulthood as the town barber, a quiet, but integral part of this small Kentucky farming community. His abiding love for a younger woman named Mattie results in an emotional commitment to her for the rest of their adult lives, even though he can never marry her. Mattie is married to an ambitious but perhaps misguided farmer whose love for farming is associated more with his own elevation as a modern businessman than a love for the land. He ultimately loses Mattie's family's land due to bad decisions. Jayber watches his beloved community deteriorate as modern, mechanized farming replaces the older, more earth-friendly methods of farmers who have passed on.

Redefining and continually examining his faith, Jayber tells the story of his quiet life as the Port William barber, the local gravedigger, and a custodian at Port Williams' church. In his old age, Jayber takes up residence in a simple cabin on the river, avoiding ever having to answer to government authorities for the lack of running water in his shop.

Jayber is a witness to wars that take many of the town's citizens, and the juggernaut force of agribusiness that changes the town forever. True to his moral code, Jayber never indulges in luxuries like running water, disturbing the earth that he loves as little as possible. His story is bittersweet, and his introspection so penetrating that it is easy to wonder whether this is Berry's novel or his autobiography.