In four sections, All the Pretty Horses introduces the coming of age tale of John Grady Cole. Chapter 1 begins in 1949 and the story starts with Cole returning from attending his grandfather's funeral. Left to his grandfather's care by parents unable to raise the boy, sixteen-year-old Cole had been living on his grandfather's ranch in San Angelo, Texas. Cole's mother is an actress of little acclaim and is set to inherit her father's ranch; a ranch she does not want. The boy's father suffers physical and mental illness resulting from the Second World War. Cole is faced with losing the only home he has ever known.

Chapter 1 of All the Pretty Horses introduces the first of many trials of John Grady Cole and his friend Lacey Rawlins. Unable to stop the sale of his grandfather's ranch, Cole sets out with seventeen-year-old Rawlins for parts unknown. On horseback, through south Texas and into Mexico, they travel in 1949. The two friends set out with stars in their eyes and myths on their minds. Shortly before crossing into Mexico, Cole and Rawlins are joined by a third youth, Jimmy Blevins, mounted on a fantastic horse and claiming a false identity. It is traveling through the small towns and rural regions of Mexico that Blevins loses his horse, pistol, and clothes during a thunderstorm. The three discover the youth's horse in a nearby town and Blevins steals the horse, pursued by armed men. Cole and Rawlins are separated from Blevins during the chase. With the addition of Blevins to their company, Cole and Rawlins have embarked upon a path that would ultimately cost them their innocence and most of their youthful ignorance.

John Grady Cole and Lacey Rawlins continue their travel south through Mexico. It is in Chapter 2, that the two friends arrive at the Hacienda de Nuestra Senora de la Purisima Concepcion. With their knowledge of ranching and Cole's talent for training horses, the two young men are employed at the hacienda for many months. During his time there, Cole becomes enamored of Alejandra, the beautiful black-haired daughter of the hacienda's owner Don Hector Rocha y Villareal. Disregarding warnings from the girl's aunt, the Duena Alfonsa, to stay away from Alejandra, Cole and Alejandra become lovers. One night, after a camp trip, John Grady Cole and Rawlins are arrested and taken from the hacienda in manacles.

It is Chapter 3 and in the town of Encantada, where John Grady Cole and Lacey Rawlins again see Jimmy Blevins. Blevins has been arrested after having attempted to steal back the pistol he had lost during the same thunderstorm in which he had lost his horse. In the process of stealing the pistol, Blevins had murdered one of the rurales, a Mexican peasant. In an attempt to ease his plight, as he had been severely beaten, Blevins had given Cole and Rawlins' names to the Mexican captain as co-conspirators in the recovery of his horse. Many areas of Mexico in the early twentieth century are still lawless. The father of the man Jimmy Blevins' killed decides the youth's fate. A payoff to the captain ensures only Cole and Rawlins survive to arrive at the Mexican prison in Saltillo. The captain on arriving at the prison informs the two young men that the only way for Cole and Rawlins to leave the prison is if arrangements are made. Americans must agree to these arrangements or they do not do so well. In the prison, they meet a political exile name Perez, who has been a prisoner for some time. Perez explains that the arrangements the captain had been referring to involve the payment of large sums of money to Perez. The money must be paid by one of the American prisoners' rich relatives or friends. This is the only manner in which an American will be freed from the prison. During prison fights, Lacey Rawlins is gravely injured and Cole kills a man to escape a similar fate. After seven weeks, the two men are released without being told why. Cole discovers that it was Alejandra's aunt who secured the release of the two young men. After their release, Cole and Rawlins buy food and clothing. After a brief discussion it is decided that Lacey Rawlins will return to his family's home in San Angelo, Texas. Cole declares his intention of returning to the Hacienda de Nuestra Senora de la Purisima Concepcion and seeing Alejandra.

Chapter 4 begins Cole's solo journey back to the Hacienda de Nuestra Senora de la Purisima Concepcion. Alejandra's aunt explains to Cole the agreement she and the young girl have made. The aunt would arrange for the young men to be released from prison if Alejandra agreed never to see Cole again. Cole travels to where Alejandra is. In their final meeting, Alejandra explains she has chosen to remain with her family and says farewell to Cole. Disheartened and disillusioned, Cole begins his journey north. He determines that he will not leave his horse in Mexico and returns to the town of Encantada. With the loss of his ideals, Cole feels he has nothing to lose and takes the captain hostage while he sets out to retrieve his and Rawlins' horses. Cole also finds and reclaims Blevins' horse that was ultimately the cause of much pain and death.

Heading into the hills with the horses and the captain, Cole is overtaken by brigands, who seize the captain. Arriving in Texas, Cole returns Rawlins' horse to his friend. Cole is taken to court attempting to return Blevins horse to its rightful owner. False charges were made by men who wanted to claim the horse themselves. Finding no owner and hearing Cole's case, a judge listens to the tale of John Grady Cole and rules in his favor. The story concludes when Cole returns to San Angelo, Texas to discover both his father and childhood nurse dead. He has arrived in time to witness her funeral. All the Pretty Horses ends as it had begun: with John Grady Cole attending a funeral. Having left Texas with his innocence and youthful ignorance intact, Cole has returned with little to his name and less of his ideals.