Markov chains: finite sink is positive recurrent

Dependencies:

  1. Markov chain
  2. Markov chains: accessibility and state classes
  3. Markov chains: positive recurrence
  4. Markov chains: positive recurrence is a class property
  5. Markov chains: long run proportion of a state
  6. Markov chains: long run proportion is inverse of time to reenter (incomplete)

Let $X = [X_0, X_1, \ldots]$ be a markov chain. Let $J$ be a state class containing a finite number of states. If no other state class is accessible from $J$, then all states in $J$ are positive recurrent.

Proof

Let $i \in J$. Since only states in $J$ are accessible from $i$, and there are a finite number of states in $J$, \[ \Pr\left(\sum_{j \in J} \pi_j = 1 \mid X_0 = i\right). \]

Let $T_j = \min_{t \ge 1} (X_t = j)$ and $m_j = \E(T_j \mid X_0 = j)$. Suppose no state in $J$ is positive recurrent. Then $\forall j \in J$, $m_j = \infty$. Hence, $\forall j \in J$, $\Pr(\pi_j = 0 \mid X_0 = i) = 1$, which implies $\Pr(\sum_{j \in J} \pi_j = 0 \mid X_0 = i)$, which is a contradiction. Hence, some state in $J$ is positive recurrent. Since positive recurrence is a class property, all states in $J$ are positive recurrent.

Dependency for: None

Info:

Transitive dependencies:

  1. /sets-and-relations/poset
  2. /sets-and-relations/equivalence-relation
  3. /sets-and-relations/de-morgan-laws
  4. /sets-and-relations/countable-set
  5. /analysis/topological-space
  6. Group
  7. Ring
  8. Semiring
  9. Matrix
  10. Identity matrix
  11. σ-algebra
  12. σ-algebra is closed under countable intersections
  13. Measure
  14. Probability
  15. Conditional probability (incomplete)
  16. Generated σ-algebra
  17. Measurable function
  18. Borel algebra
  19. Generators of the real Borel algebra (incomplete)
  20. Random variable
  21. Markov chain
  22. Markov chains: long run proportion of a state
  23. Markov chains: recurrent states
  24. Markov chains: positive recurrence
  25. Chapman-Kolmogorov equation
  26. Markov chains: accessibility and state classes
  27. Markov chains: long run proportion is inverse of time to reenter (incomplete)
  28. Markov chains: positive recurrence is a class property