Russell Baker's memoir describes his childhood in rural Virginia, his youth growing up in the Great Depression, and his young adulthood in Baltimore with his mother. Russell's reminiscences are centered on his relationship with his mother, a single parent through much of his youth, who eventually grows senile and is unable to tell him about her own life growing up.

Lucy is the daughter of a gentile Virginia lawyer who dies unexpectedly, leaving Lucy to drop out of college and take teaching jobs in rural areas. She meets Benny Baker, a son of a large local family, and gets pregnant. The two marry over objections by Benny's matriarchal mother Ida Rebecca. They have three children, Russell, Doris, and little Audrey, and Lucy fails to reform Benny from drinking. Benny is diabetic, and he dies an early death, leaving Lucy with three children. One of Benny's brothers adopts Audrey, and Lucy moves in with her brother Allen. The Great Depression begins, and Lucy can't get a job.

Allen takes in Lucy, Russell, and Doris, as well as his brother Charlie. His fast-talking, money-grubbing brother Hal soon joins them, and he gets Lucy to give him $75 of her hard-earned money to start a lumber business that never happens. Lucy takes what little she has left to move out to Baltimore. In Baltimore, Russell takes a paper route and learns to get along in the big city. He's embroiled in fights, but he's good at school. Meanwhile, his mother tries to make a living, selling magazine subscriptions for commission. At one point, they have to get government handouts to eat.

Eventually, Lucy meets and marries Herb Orrison, who works on the railroad and makes enough money to support her family. Russell hates his stepfather, who has replaced him as the man of the house. Lucy has another daughter, and the family moves to a house. Finally, Lucy has had the home of her own that she's dreamed about. Russell is about to graduate high school, but he can't afford college and doesn't know what he'll do for a job. A friend tells him about scholarships offered by Johns Hopkins University, and Russell gets accepted, with a scholarship to pay his tuition.

During World War II, Russell drops out of college to become a pilot in the Navy, but just as he's finishing his training, the war ends. Russell comes back and eventually finishes college and goes to work for the Baltimore Sun. Meanwhile, he meets Mimi, a modern girl. His mother disapproves, and Russell doesn't think she's the right kind of girl to marry. Still, he can't live without her, and after four years they marry. Years pass; his mother grows older. Herb dies. Russell has children and grandchildren. In the end, his mother is confined to a nursing home, where in her senility, she relives the days of her past in a world of her own.