The story begins with a strange man walking out of the Australian wilderness, or bush, into an English settlement in the early 1860's. The settlers are suspicious of the visitor. Even though he proves he is in fact, English, he was raised by black natives and is therefore viewed by some as untrustworthy. The man, Gemmy, speaks limited English and makes some notable friends in the settlement.

The McIvor family decides to take in the visitor. The father, Jock, although reluctant at first, does grow to like Gemmy. The oldest daughter Janet also has a strong bond with him. The "adopted" cousin, Lachlan, who does not get along well with Janet, also has a friendship with Gemmy and acts as his protector.

The minister, Mr. Frazer, finds that Gemmy's knowledge is invaluable in his botany work and they work to categorize the various kinds of plants and fruits in the area. Gemmy shows him what uses the different plants have and which ones are edible. Mr. Frazer uses the information to write a report showing how the colonists can make use of the native crops. Eventually he presents the information he has gathered to the governor, but is rebuffed.

Several other settlers don't like Gemmy and see him as a "white black" and therefore a threat to their livelihood. In keeping a close watch on him, they see a meeting between Gemmy and some natives. Their interpretation of this is that Gemmy is a spy for the natives. This event puts a serious strain between the McIvors and especially Jock, who protects Gemmy, and the rest of the settlers. Eventually Gemmy is attacked and is rescued by Jock. For his own safety, Gemmy moves further out of town to live..

Gemmy believes that the papers used to record his story upon initially entering the village contain his magic and must be returned to him. He feels his energy slipping by being so far away from his native home. He gets back the papers he believes to be his, and walks back into the bush.

Years later, Janet and Lachlan reminisce about their times in the settlement and reflect how much Gemmy brought to their lives.