One of the most basic paradigms in distributed computing is that of message passing. For computing nodes in a distributed system to be able to communicate effectively and share their work and information, it is necessary for them to either be able to write to and read from some common area of memory, or else for nodes to be able to send messages to one another. A distributed system that uses the first method is said to be following the shared-memory model, while one that uses the second is said to be using message-passing. The shared memory model is used in parallel processing with a small number of processors, while the message-passing model is used in larger systems with many processors.

In a message-passing system, nodes or processors communicate by sending messages over links or communication channels which provide bidirectional connectivity between pairs of nodes. The pattern of...