"the state sequence" associated to . Definition of the jump matrix Π The jump matrix , alternatively written if we wish to emphasize the dependence on , is the matrix where is the zero set of the function Jump-chain/holding-time property We say is Markov with initial distribution and rate matrix to mean: the trajectories of are almost surely right continuous, let be a modification of to have (everywhere) right-continuous trajectories, almost surely (note to experts: this condition says is non-explosive), the state sequence is a discrete-time Markov chain with initial distribution (jump-chain property) and transition matrix and (holding-time property). Infinitesimal definition We say is Markov with initial distribution and rate matrix to mean: for all and for all , for all and for small strictly positive values of , the following holds for all such that : , where the little-o term depends in a certain way on . The above equation shows that can be seen as measuring how quickly the transition from to happens for , and how quickly the transition away from happens for . Properties Communicating classes Communicating classes, transience, recurrence and positive and null recurrence are defined identically as for discrete-time Markov chains. Transient behaviour Write P(t) for the matrix with entries pij = P(Xt = j | X0 = i). Then the matrix P(t) satisfies the forward equation, a first-order differential equation where the prime denotes differentiation with respect to t. The solution to this equation is given by a matrix exponential In a simple case such as a CTMC on the state space {1,2}. The general Q matrix for such a process is the following 2 × 2 matrix with α,β > 0 The above relation for forward matrix can be solved explicitly in this case to give However, direct solutions are complicated to compute for larger matrices. The fact that Q is the generator for a semigroup of matrices is used. Stationary distribution The stationary distribution for an irreducible recurrent CTMC is the probability distribution to which the process converges for large values of t. Observe that for the two-state process considered earlier with P(t) given by as t → ∞ the distribution tends to Observe that each row has the same distribution as this does not depend on starting state. The row vector may be found by solving with the additional constraint that Example 1 The image to the right describes