Rich Man, Poor Man is the epic tale of one family; a tale that takes place from the turmoil of World War II to the beginning of Vietnam. There are three children in the Jordache family: Gretchen, the oldest, Rudolph, the responsible one, and Thomas, the troublemaker. The novel follows the lives of each member of the family, through happiness and turmoil, separation and reunions, and takes the reader on an emotional roller coaster that they will not soon forget.

Rudolph Jordache has a crush on his French teacher. When he is not writing love letters to her in French, he is drawing naked pictures of what he imagines he will see when he finally convinces her of the depth of his love. However, when the French teacher confiscates one of Rudolph's drawings, he is forced to reveal to his father his fantasies. To Rudolph's surprise and amazement, his father stands up for him, revealing the true character of the woman Rudolph though he loved.

Gretchen is caught out on a country road about to rendezvous with two patients from the VA hospital where she volunteers. The man who has caught her, Teddy Boylan, takes her to lunch and quickly becomes her lover. Gretchen enjoys her time with Boylan, but does not want a commitment to a man who is cruel and much older than she. Little does Gretchen know, however, her secret affair has been discovered by her brother Thomas one night after a friend of Thomas's spots Gretchen and Teddy together. Thomas, enraged by the defamation of his sister, burns a cross on Teddy's lawn the night of VE day.

When Axel Jordache, the patriarch of the family, learns of Thomas's actions, Thomas is sent away to live with an uncle in Ohio. Gretchen takes this opportunity to run away, taking what little money she has and moving to New York. In New York, Gretchen meets the man of her dreams and marries him despite Teddy Boylan's continued affections. Rudolph, who has become a friend of Boylan's, is the bearer of bad news. Teddy does not take his jealousy out on Rudolph. In fact, Boylan pays Rudolph's way through college after Axel is forced to use the money he saved for Rudolph in order to rescue Thomas from jail.

Rudolph puts his education to good use, becoming manager of Calderwood's Department Store. After Rudolph meets Johnny Heath, a Wall Street broker at a party at Gretchen's house, the two put their heads together and come up with a business plan to create a multimillion-dollar company with Duncan Calderwood, the owner of Calderwood's Department Store. The plan is so successful that Rudolph becomes a millionaire. However, the work has occupied his time for so long that Rudolph has no personal life until he meets Jean.

Jean is a photographer who was hired to take Rudolph's picture at the opening of a new strip mall. Rudolph falls madly in love with her and after some stops and starts, manages to coax her to marry him. Meanwhile, Thomas has reappeared in New York as a headlining boxer. When Gretchen and Rudolph visit him after the fight, they learn he is married and has a son, Wesley. Gretchen, too, has divorced her first husband and has married a movie director. However, while in New York to settle her son in boarding school, Gretchen's second husband is killed in a car accident.

Thomas's boxing career is short lived, however, after a fight in London. Thomas had taken his earnings from this fight and had blown them on an English girl on the beaches of France. This indulgence causes Thomas to fall out of shape and lose a major fight, effectively ruining his career. While in Las Vegas working as a sparring partner for another boxer, Thomas gets into a fight with the other boxer, breaking the man's jaw. Afraid that the mob will want revenge, Thomas flees to New York where he takes work on board a cargo ship. It is during this time that Thomas discovers his true love, the sea.

Thomas schemes with a man he met on board the cargo ship to buy a boat of their own, but they need money. Shortly after his mother's death, Thomas is surprised to learn that Rudolph has taken some money Thomas has given him to replace his college fund and has invested it. There is now more than enough capital for Thomas to buy his boat.

Gretchen has found a new career, cutting film for the movie industry, and a new love, another director. Although Gretchen's son, now in college, does not approve of her boyfriend, Gretchen has found contentment. Rudolph has become mayor of the small town where he has lived since college. However, his political career is cut short due to a bad end to a standoff at the local college. Rudolph's wife, Jean, has also succumbed to alcoholism after the tragic end to her second pregnancy.

Thomas has the life everyone dreams of, a boat, good friends, his son, and a new love. Thomas invites Gretchen and Rudolph to visit him on his boat for his wedding. After the wedding, Thomas is awakened in the middle of the night and told that Jean is in trouble. Thomas goes to the bar where Jean snuck off to and finds her being assaulted by a local pimp. Thomas fights the man off and takes Jean home. However, the pimp finds Thomas several days later and Thomas dies from the injuries he sustains from the resulting fight.