ions. If the process is initiated at time t1 with a stationary solution, then the process is also stationary, meaning that the probability distribution p(x) at time tn+1 is the same as at time tn or tn-1 or t1. Note that such a stationary solution is not necessarily unique. Two conditions for uniqueness of the stationary solution,1 which we will assume here without demonstration, are: (a) The process is aperiodic (i.e., the evolution of p(x) does not show periodic oscilla- tions with equal or increasing amplitudes); (b) There exists a nonzero probability that the variable X E X will be reached within a finite number of steps (the process is then said to be irreducible). T.M. Cover and J.A. Thomas, Elements of Information Theory (New York: John Wiley & Sons, 1991), Ch. 2. Appendix D Under these two conditions, the stationary solution is unique. Moreover, the distri- bution at time tn in the limit n 8 asymptotically converges towards the stationary solution, regardless of the initial distribution at time t1. This property will be demon- strated in the second part of this appendix. Assuming that the conditions of uniqueness are satisfied in the previous example, the entropy H(X)t=t converges towards the limit: H(X){=1x = Hoo (D14) It is easily verified that when the stationary solution is uniform (B = M/2), then Hx = Hmax = log 2 = 1 bit/symbol, which represents the maximum possible entropy for a two-state distribution (Chapter 4). In the general case where the stationary solution is nonuniform (B # M/2), we have, therefore, Hoo < Hmax. This means that the system evolves towards an entropy limit that is lower than the maximum. Here comes the inter- esting conclusion for this first part of the appendix: assuming that the initial distribution is uniform and the stationary solution nonuniform, the entropy will converge to a value Hoo < Hmax = H(X)t=t1. This result means that the entropy of the system decreases over time, in apparent contradiction with the second law of thermodynamics. Such a contradiction is lifted by 