Markov chains: recurrent class is sink

Dependencies:

1. Markov chain
2. Markov chains: recurrent states
3. Markov chains: accessibility and state classes
4. Markov chains: recurrence is a class property

$\newcommand{\ihat}{\widehat{\imath}}$ $\newcommand{\khat}{\widehat{k}}$ Let $X = [X_0, X_1, \ldots]$ be a markov chain with transition function $P$. Let $J$ be a state class that is accessible from another state class $I$. Then $I$ isn't a recurrent class.

Proof

$J$ is accessible from $I$ implies that for some $i \in I$ and $j \in J$, there exists a sequence $Q$ of states from $i$ to $j$ such that each transition in the sequence has positive probability. Let $\ihat$ be the last state from $I$ in $Q$. Let $\khat$ be the state after $\ihat$ in $Q$. Then $\khat$ belongs to state class $K \neq I$. Let $q = P(\ihat, \khat)$. Then $q > 0$.

Since accessibility of state classes is anti-symmetric, $\ihat$ is not accessible from $\khat$. Hence, for all $t \ge 0$, $P^{(t)}(\khat, \ihat) = 0$.

\begin{align} & \Pr(\textrm{get back to } \ihat \mid X_0 = \ihat) \\ &= \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_0 = \ihat\right) \\ &= \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 = \khat, X_0 = \ihat\right) \Pr(X_1 = \khat \mid X_0 = \ihat) \\ &\quad+ \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 \neq \khat, X_0 = \ihat\right) \Pr(X_1 \neq \khat \mid X_0 = \ihat) \\ &= q\Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_0 = \khat\right) \tag{by markov property} \\ &\quad+ (1-q)\Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 \neq \khat, X_0 = \ihat\right) \\ &\le q\sum_{t=1}^{\infty} \Pr(X_t = \ihat \mid X_0 = \khat) + (1-q) \\ &= q\sum_{t=1}^{\infty} P^{(t)}(\khat, \ihat) + (1-q) \\ &= (1 - q) < 1. \end{align}

Since probability of getting back to $\ihat$ from $\ihat$ is less than 1, $\ihat$ is not recurrent. Since recurrence is a class property, $I$ is not recurrent.

Dependency for: None

Info:

• Depth: 10
• Number of transitive dependencies: 32

Transitive dependencies:

1. /sets-and-relations/poset
2. /measure-theory/linearity-of-lebesgue-integral
3. /measure-theory/lebesgue-integral
4. /sets-and-relations/equivalence-relation
5. /sets-and-relations/de-morgan-laws
6. /sets-and-relations/countable-set
7. /analysis/topological-space
8. Group
9. Ring
10. Field
11. Vector Space
12. Semiring
13. Matrix
14. Identity matrix
15. σ-algebra
16. σ-algebra is closed under countable intersections
17. Measure
18. Probability
19. Conditional probability (incomplete)
20. Generated σ-algebra
21. Measurable function
22. Borel algebra
23. Generators of the real Borel algebra (incomplete)
24. Random variable
25. Markov chain
26. Markov chains: recurrent states
27. Chapman-Kolmogorov equation
28. Markov chains: accessibility and state classes
29. Expected value of a random variable
30. Linearity of expectation
31. Markov chains: recurrent iff expected number of visits is infinite
32. Markov chains: recurrence is a class property