Markov chains: recurrent class is sink

Dependencies:

  1. Markov chain
  2. Markov chains: recurrent states
  3. Markov chains: accessibility and state classes
  4. Markov chains: recurrence is a class property

Let $X = [X_0, X_1, \ldots]$ be a markov chain with transition function $P$. Let $J$ be a state class that is accessible from another state class $I$. Then $I$ isn't a recurrent class.

Proof

$J$ is accessible from $I$ implies that for some $i \in I$ and $j \in J$, there exists a sequence $Q$ of states from $i$ to $j$ such that each transition in the sequence has positive probability. Let $\ihat$ be the last state from $I$ in $Q$. Let $\khat$ be the state after $\ihat$ in $Q$. Then $\khat$ belongs to state class $K \neq I$. Let $q = P(\ihat, \khat)$. Then $q > 0$.

Since accessibility of state classes is anti-symmetric, $\ihat$ is not accessible from $\khat$. Hence, for all $t \ge 0$, $P^{(t)}(\khat, \ihat) = 0$.

\begin{align} & \Pr(\textrm{get back to } \ihat \mid X_0 = \ihat) \\ &= \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_0 = \ihat\right) \\ &= \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 = \khat, X_0 = \ihat\right) \Pr(X_1 = \khat \mid X_0 = \ihat) \\ &\quad+ \Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 \neq \khat, X_0 = \ihat\right) \Pr(X_1 \neq \khat \mid X_0 = \ihat) \\ &= q\Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_0 = \khat\right) \tag{by markov property} \\ &\quad+ (1-q)\Pr\left(\bigvee_{t=1}^{\infty} (X_t = \ihat) \mid X_1 \neq \khat, X_0 = \ihat\right) \\ &\le q\sum_{t=1}^{\infty} \Pr(X_t = \ihat \mid X_0 = \khat) + (1-q) \\ &= q\sum_{t=1}^{\infty} P^{(t)}(\khat, \ihat) + (1-q) \\ &= (1 - q) < 1. \end{align}

Since probability of getting back to $\ihat$ from $\ihat$ is less than 1, $\ihat$ is not recurrent. Since recurrence is a class property, $I$ is not recurrent.

Dependency for: None

Info:

Transitive dependencies:

  1. /analysis/topological-space
  2. /sets-and-relations/countable-set
  3. /sets-and-relations/poset
  4. /sets-and-relations/equivalence-relation
  5. /sets-and-relations/de-morgan-laws
  6. /measure-theory/linearity-of-lebesgue-integral
  7. /measure-theory/lebesgue-integral
  8. σ-algebra
  9. Generated σ-algebra
  10. Borel algebra
  11. Measurable function
  12. Generators of the real Borel algebra (incomplete)
  13. Measure
  14. σ-algebra is closed under countable intersections
  15. Group
  16. Ring
  17. Field
  18. Vector Space
  19. Semiring
  20. Matrix
  21. Identity matrix
  22. Probability
  23. Conditional probability (incomplete)
  24. Random variable
  25. Markov chain
  26. Markov chains: recurrent states
  27. Chapman-Kolmogorov equation
  28. Markov chains: accessibility and state classes
  29. Expected value of a random variable
  30. Linearity of expectation
  31. Markov chains: recurrent iff expected number of visits is infinite
  32. Markov chains: recurrence is a class property