Jack spends a day in Port William, his home since birth, reflecting on his life, from the time he was a young boy born in the wake of the Civil War through an unhappy marriage to his final days as an old man who has been left behind by a society that has reinvented its values. Jack's first memories, which he recalls at the beginning of a new day, are of a young child in a family that had been torn apart by the Civil War. His two older brothers, whom he barely knew, had died, and his grief-stricken mother died before the war came to an end. His father, too, died a kind of spiritual death, and Jack was left to be raised by his older sister, Nancy, and her eventual husband Ben Feltner. Ben was everything Jack ever wanted to be, and even in his old age, Jack looked back on Ben as the model of manhood and strength.

From a young age, Jack developed a strong work ethic, in part because it allowed him an escape from his troubled household. His father died when he was still a young man and he inherited the farm. He took to caring for it with zeal and it almost became an extension of his being. As he grew older, he increasingly desired the companionship of a wife. He thought to find it in the beautiful Ruth and after a short courtship the two were married. Immediately, however, troubles began. Ruth saw Jack as a kind of project. She wanted him to be an upright, God-fearing, ambitious, wealthy man, and Jack was happy with what he had. Desperate to fulfill her expectations, or at least distract himself with more work, Jack bought more land and hired the help of a black man named Will Wells to tend to it. Before long, the inequity of the relationship between Jack and Will is too much for Jack to bear; Jack cannot stand knowing that all the work Will has put in is, ultimately, for Jack's gain. Without the help necessary to use the field it falls into disuse and Jack is forced to sell it, to the disappointment of his wife. There is now a division between them that will never be bridged, and they cease to have any substance to their marriage after the birth of their daughter, Clara, who Ruth jealously keeps away from Jack.

Jack's reverie is interrupted by a big dinner at the house of Mat Feltner, Ben's son, and the reader is given a picture of the quaint, though fading Southern culture. Most of the people there are quite old and Andy, the youngest, is headed away to college in the city, a sign of the decline of the South. Jack cannot remain long in the present, however, and after dinner he continues to recount his life.

Heartbroken and unhappy, Jack is swept up into an affair with the widow of a doctor's wife, Rose. For a while, their passionate romance is everything Jack had always wanted from Ruth, but when he sees that Ruth is hurt, guilt begins to overwhelm joy. Before he makes a decision to formally end his affair with Rose, she dies in a tragic accident while is he out of town on business. After that, Jack resigns himself to working on his farm, and with great pride is able to pay off the entire debt on the farm. Their daughter was growing up, however, and he had no rest from his work. His wife, whose own dreams had ended in disappointment, wanted to make sure the same did not happen to Clara, and the family scrounged for all the money they could to send her away to college. While there, she lost touch with much of her upbringing and Port William and shortly after her graduation married a wealthy banker, Glad Pettit, and lived with him in a large house in Louisville. The Pettits would visit home frequently while Ruth still lived, but after her death their visits were rarer and motivated more by etiquette than any real bond.

Eventually, Jack became too old to live on his own, and at the urging of his friends, he decided to pass his final years living in the town's hotel. With no family around and no work, he lost himself to his memories, which is where the story finds him at its outset. Gazing out of his window over the land that taught him all of his values, Jack dies. He is given a simple burial—though, perhaps, not simple enough, in the eyes of those who knew him best—and the town returns to work. They do not forget, however, Old Jack, who for them stands as a symbol of the virtues of the way things used to be.