# A conceptual doubt on steady state probability of a DTMC

Consider a DTMC with state space $\{1, 2,\ldots {}{} \}$. Now we want to calculate the probability that state 1 is followed by state 2 in the long run i.e, $P(X_n=1, X_{n+1}=2)$ as $n$ tends to infinity. Now if we evaluate this expression then it comes as $\pi_{1}p_{12}$. I have basic conceptual doubt here. If a DTMC reaches state 1 in the long-run then it will stay there for ever. Then the required probability should be zero. I know that I am making a mistake, but not able to find where.

-
What makes you think it will stay there forever? It doesn't (necessarily) stay there forever if it starts there, after all. –  Jonathan Christensen Jan 11 '13 at 1:24
You should probably try to state rigorously what is meant by the phrase "in the long run". –  Nate Eldredge Jan 11 '13 at 1:54
Indeed, if $P(X_n=1)\to\pi(1)$ then $P(X_n=1,X_{n+1}=2)\to\pi(1)p(1,2)$. –  Did Jan 11 '13 at 11:13

This is vague and, most likely, incorrect. The word "steady" in "steady-state analysis" refers not to the value of $X_n$ being steady, but to the probabilities $P(X_n=k)$, for each fixed $k$. So, the probability of $P(X_n=1)$ may converge to $\pi(1)$ but this does not mean that $X_n$ will stay at $1$. As Did remarked, in this scenario $P(X_n=1,X_{n+1}=2)\to\pi(1)p(1,2)$ as $n\to\infty$.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.9067155122756958, "perplexity": 278.5004433976823}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2015-35/segments/1440644065464.19/warc/CC-MAIN-20150827025425-00165-ip-10-171-96-226.ec2.internal.warc.gz"}