<P> Note that there is no definitive agreement in the literature on the use of some of the terms that signify special cases of Markov processes . Usually the term `` Markov chain '' is reserved for a process with a discrete set of times , i.e. a discrete - time Markov chain ( DTMC ) , but a few authors use the term `` Markov process '' to refer to a continuous - time Markov chain ( CTMC ) without explicit mention . In addition , there are other extensions of Markov processes that are referred to as such but do not necessarily fall within any of these four categories ( see Markov model ) . Moreover , the time index need not necessarily be real - valued ; like with the state space , there are conceivable processes that move through index sets with other mathematical constructs . Notice that the general state space continuous - time Markov chain is general to such a degree that it has no designated term . </P>
<P> Note that there is no definitive agreement in the literature on the use of some of the terms that signify special cases of Markov processes . Usually the term `` Markov chain '' is reserved for a process with a discrete set of times , i.e. a discrete - time Markov chain ( DTMC ) , but a few authors use the term `` Markov process '' to refer to a continuous - time Markov chain ( CTMC ) without explicit mention . In addition , there are other extensions of Markov processes that are referred to as such but do not necessarily fall within any of these four categories ( see Markov model ) . Moreover , the time index need not necessarily be real - valued ; like with the state space , there are conceivable processes that move through index sets with other mathematical constructs . Notice that the general state space continuous - time Markov chain is general to such a degree that it has no designated term . </P>
markov chains with finite and countable state space
markov chains with finite and countable state space
a discrete - time Markov chain ( DTMC )
a discrete - time Markov chain ( DTMC )