timehomogeneous transition density initial distribution the index in xt is usually seen as "timeor "stepparameter the index set { in the definition above was chosen out of convenience it can be replaced by any countable index set we restrict ourselves to time-homogeneous markov chains -markov chains for which the conditional pdfs fxt+ xt ( xdo not depend on twe abbreviate these as ( xthe { ( )are called the (one-steptransition densities of the markov chain note that the random variables or vectors {xt may be discrete ( taking values in some set { }or continuous ( taking values in an interval [ or rd in particularin the discrete caseeach ( xis probabilityq( xp[xt+ xt xthe distribution of is called the initial distribution of the markov chain the onestep transition densities and the initial distribution completely specify the distribution of the random vector [ xt ]namelywe have by the product rule ( and the markov property that the joint pdf is given by fx ,xt ( xt fx ( fx ( fxt xt- , (xt xt- fx ( fx ( fxt xt- (xt xt- fx ( ( ( (xt xt- ergodic limiting pdf global balance equations markov chain is said to be ergodic if the probability distribution of xt converges to fixed distribution as ergodicity is property of many markov chains intuitivelythe probability of encountering the markov chain in state at time far into the future should not depend on the tprovided that the markov chain can reach every state from any other state -such markov chains are said to be irreducible -and does not "escapeto infinity thusfor an ergodic markov chain the pdf fxt (xconverges to fixed limiting pdf (xas irrespective of the starting state for the discrete casef (xcorresponds to the long-run fraction of times that the markov process visits under mild conditions (such as irreducibilitythe limiting pdf (xcan be found by solving the global balance equationsp (yq( yf (xf (yq( ydy reversible reverse markov chain (discrete case)(continuous case( for the discrete case the rationale behind this is as follows since (xis the long-run proportion of time that the markov chain spends in xthe proportion of transitions out of is (xthis should be balanced with the proportion of transitions into state xwhich is (yq( yone is often interested in stronger type of balance equations imagine that we have taken video of the evolution of the markov chainwhich we may run in forward and reverse time if we cannot determine whether the video is running forward or backward (we cannot determine any systematic "looping"which would indicate in which direction time is flowing)the chain is said to be time-reversible or simply reversible although not every markov chain is reversibleeach ergodic markov chainwhen run backwardsgives another markov chain -the reverse markov chain -with transition densities ( xf (yq( ) (xto see thisfirst observe that (xis the long-run proportion of time spent in for both the original and reverse markov chain secondlythe "probability fluxfrom to in the reversed chain must be equal to the probability flux from to in the original chainmeaning (xe ( xf (yq( )which yields the