served at time t = t2, and SO on, until observation time t = tn. Then p(x1, X2,..., Xn) is the probability of observing X1, then x2, etc., until Xn. If we repeat the observation of the n events, but now starting from any time tq(q > 1), we shall obtain the series labeled 1x1+q...Xntq, which corresponds to the joint distribution p(x1+q, xn+q). By definition, the stochastic process is said to be stationary if for any q we have p(x1+q,x2+q,.. xn+q) = meaning that the joint distribution is invariant with time translation. Note that such an invariance does not mean thatx1+q=x1,x2+q=x2,and SO on! The property only means that the joint probability is time invariant, or does not depend at what time we start the observation and which time intervals we use between two observations. What is a Markov process? Simply defined, it is a chain process where the event outcome at time tn+1 is only a function of the outcome at time tn, and not of any other preceding events. Such a property can be written formally as: This means that the event Xn+1 is statistically independent, in the strictest sense, from all preceding events but Xn. Using Bayes's formula and the above property, Appendix D we get: p(x1,x2) = p(x2|x1)p(x1) = p(x1,X2,x3)=p(x3/x1,x2)p(x1, = x2) = and consequently A Markov chain is said to be time invariant if the conditional probabilities p(xn/xn-1) do not depend on the time index n, i.e., they are themselves time invariant. For instance, if a and b are two specific outcomes, we have p(xn = b|xn-1 = a) = p(xn-1 = b|xn-2 = a) ==px2=bx1=a). = If we recall the property of conditional probabilities: then we have for time-invariant Markov chains (as applying to any time tn+1): or equivalently p(xn+1) where we define Pxnxn+1 = p(xn+1/Xn) as being the coefficients of a certain transition matrix P (note the reverse order of the coefficient subscripts). Such a transition matrix uniquely defines the Markov chain, and defines the evolution of any other probability distribution q, namely: The expressions in Eqs. (D7) or (D8) corre