Jim Glass is turning ten years old as the story opens. His mother's name is Cissy and the two of them live with one of Cissy's brothers. Cissy's other two brothers live nearby in the town of Aliceville. Jim's father, who was also named Jim, died a week before Jim was born.

He rises with his uncles and is pleased when he's allowed to go with Uncle Zeno and Uncle Al to the fields that day. Once there, Jim is amazed at how difficult the hoeing is. He's soon bored and chops down a couple of cotton stalks. When his Uncle Zeno asks Jim about them, Jim lies. Uncle Zeno uses the situation as a lesson and Jim, knowing he's disappointed his uncle, goes home. There, he encounters an adult friend named Ralph Whiteside, better known as Whitey, who is a feed salesman. Whitey and Jim look over the building that will be the new school and Whitey gives Jim a baseball for his birthday. The day passes and Jim believes he hasn't gotten any birthday gifts other than the baseball. Then his mother produces a birthday cake, complete with candles, and Jim finds that his uncles and mom have bought him a baseball bat and glove.

When school begins, Jim and his friends are suddenly thrown into a class with boys from Lynn's Mountain because several small districts have consolidated. Jim and a boy named Penn Carson quickly emerge as the best in sports and the leaders of the "town boys" and the "mountain boys." When the two teams face off for their first game of baseball, the score is tied and Penn is headed for home. Jim throws the ball and strikes Penn in the back. Penn is angry but some of the boys point out that Penn is a Quaker and, as such, doesn't believe in fighting. Jim can't decide even for himself why he'd hit Penn with the ball. The stage is set for a rivalry that continues for some time though the boys do pair off when they happen to meet up in the nearby town of New Carpenter.

Jim and Penn learn that a famous baseball player is aboard a train that's stopped in their small town and the boys begin playing catch near the train, hoping the man will take note of how good they are. The boys argue over who should have Jim's glove and Jim eventually grows so angry that he throws too hard and the ball lands in a puddle. As Penn turns to retrieve it, Penn falls and is unable to get up. It's soon discovered that he has polio and the town is quarantined. Jim is housebound until it's certain he doesn't have the disease as well.

Jim and his uncles go onto Lynn's Mountain to visit Penn. While they're there, they also go to visit Jim's grandfather, Amos, who Jim has never met and who is nearing death. Jim sees the old man, but only through a window and his grandfather is unable to grasp Jim's identity. As Jim turns away from his grandfather's house, he's struck by how big the world is. He points it out to his uncles, adding that he's only one small boy. Jim's Uncle Zeno responds that Jim is right, but "You're our boy."