The Memory Palace of Matteo Ricci concerns the missionary efforts of the late 16th century Jesuit priest, Matteo Ricci, to convert the Chinese people to Christianity. Matteo Ricci is born in Macerata, Italy on October 6th, 1552 within the "papal domain" or the Papal States in what is today Italy. In 1561 he begins his education at a Jesuit school and in 1568 he leaves for Rome to study law. On August 15th, 1571, he joins the new Catholic order, The Society of Jesus and becomes a Jesuit, after which he studies at a Jesuit college in Florence and then in Rome. During this time, the Jesuit Order is young and it encourages its priests to travel across the world and convert the peoples of earth to Catholicism. So Ricci decides to become a missionary and sails to Macao, China. For the next twenty-eight years, Ricci attempts to convert the Chinese people to Catholicism, and it is during this period that most of the book takes place. Ricci believes that he can convert the Chinese elites by impressing them with his learning. During his Jesuit training, he learns to use mnemonic devices to commit vast stretches of text into memory.

Ricci's most important task is communicating the basic Christian idea to the Chinese in a way they can appreciate. It is for this purpose that he turns to the mnemonic devices he learns as a student. He teaches the Chinese to create "memory palaces," or imaginary buildings that will store images and pictures in certain locations, enabling them storage of large amounts of information in the form of visual symbols organized within an imagined structure. To communicate the Christian idea to the Chinese, he creates a memory palace for them with a reception hall that contains four images derived from Chinese ideographs that are in turn associated with four pictures representing stories and individuals in the Bible. Each image has a caption and three have his commentaries. The pictures are associated with these Chinese ideographs in order to summarize Christian ideas for the Chinese people.

The Memory Palace of Matteo Ricci is structured around the idea of the memory palace and the four pictures and four images that Ricci places in the memory palace of his Chinese students to help them understand Christianity. However, the ideas embodied in each image and picture are used by the author to give the relevant historical and intellectual background of Ricci's life and ideas, along with the history of his life and experiences prior to and during his stay in China. Thus, the nine chapters of the book introduce the memory palace, the four images and four pictures, but also say much, much more about life and culture in China and Europe in the last 16th and early 17th centuries, particularly within the Jesuit Order and related groups.