(3.8) If an equilibrium distribution exists the distribution of the process will always converge to it as in Equation 3.9, and, having this (apple (3.5) property, it must be unique. (j) can be regarded as the long run average proportion of time the process spends in state j. X(s)jx()kr(j) (3.9) 3.2.2 Markov processes and Markov chains As far as the Markov processes here are concerned, they may be Markov process models underlay a great deal of queuing analysis regarded as arising in the following way. The construction is based and a grasp of these models is essential to understanding analytical on the exponential which was discussed in the previous section. In models of queues. A stochastic process X(s) defined on a countable each state j the rate at which transitions are made to state k is q(j,k). state space K is said to be Markov if, given a finite sequence of n Thus the probability that the process enters state k in the next > 1 points (nearly always points in time) SI <82 < Sn. Equation interval of length dt is q(j,k)dt. The probability that a transition out 3.6 holds. of state j will take place at all in the time interval dt is therefore given by Equation 3.10. (3.10) Prx(s(36) Figure 3.6 A coin tossing chain with period 2 Models based on the exponential, Markov process and Markov chains 3/7 Mk-1 Mk Figure 3.7 A linear birth-death model Put another way, each visit to j has a duration which is exponential nificance of the local balance equations is that they show that the with rate parameter q(j). The probability that the process then enters linear birth death process is reversible in equilibrium. In crude state k is given by Equation 3.11. terms, reversible Markov processes cannot be distinguished, statis- p(j.k)=9lj.k) (3.11) tically, going forwards in time from going backwards in time. From this it is clear that a Markov process must be in equilibrium to be The equilibrium distribution, if it exists, satisfies Equation 3.12, reversible.) which are referred to as the global balance equations. J (j) ( ( , k)