A Markov chain is a process, invented by the Russian mathematician Andrei Markov (1856-1922), used for predicting future outcomes or "states" of a system based upon the current state of the system. More formally, a Markov chain is a probabilistic dynamical system consisting of a finite number of states in which the probability that the system will be in a given state at time n depends only upon the state of the system at time n-1. The states of the system form the "chain." The movement of the system from one state to the next is called a transition. Every Markov chain has a "transition matrix" which contains the probabilities involved in moving from one state to the next. Also associated with each Markov chain is an initial state vector giving the state of the system at time 0. Beginning with the initial state vector, succeeding states...