Markov chains: finite sink is positive recurrent

Dependencies:

  1. Markov chain
  2. Markov chains: accessibility and state classes
  3. Markov chains: positive recurrence
  4. Markov chains: positive recurrence is a class property
  5. Markov chains: long run proportion of a state
  6. Markov chains: long run proportion is inverse of time to reenter (incomplete)

Let $X = [X_0, X_1, \ldots]$ be a markov chain. Let $J$ be a state class containing a finite number of states. If no other state class is accessible from $J$, then all states in $J$ are positive recurrent.

Proof

Let $i \in J$. Since only states in $J$ are accessible from $i$, and there are a finite number of states in $J$, \[ \Pr\left(\sum_{j \in J} \pi_j = 1 \mid X_0 = i\right). \]

Let $T_j = \min_{t \ge 1} (X_t = j)$ and $m_j = \E(T_j \mid X_0 = j)$. Suppose no state in $J$ is positive recurrent. Then $\forall j \in J$, $m_j = \infty$. Hence, $\forall j \in J$, $\Pr(\pi_j = 0 \mid X_0 = i) = 1$, which implies $\Pr(\sum_{j \in J} \pi_j = 0 \mid X_0 = i)$, which is a contradiction. Hence, some state in $J$ is positive recurrent. Since positive recurrence is a class property, all states in $J$ are positive recurrent.

Dependency for: None

Info:

Transitive dependencies:

  1. /analysis/topological-space
  2. /sets-and-relations/countable-set
  3. /sets-and-relations/poset
  4. /sets-and-relations/equivalence-relation
  5. /sets-and-relations/de-morgan-laws
  6. σ-algebra
  7. Generated σ-algebra
  8. Borel algebra
  9. Measurable function
  10. Generators of the real Borel algebra (incomplete)
  11. Measure
  12. σ-algebra is closed under countable intersections
  13. Group
  14. Ring
  15. Semiring
  16. Matrix
  17. Identity matrix
  18. Probability
  19. Conditional probability (incomplete)
  20. Random variable
  21. Markov chain
  22. Markov chains: long run proportion of a state
  23. Markov chains: recurrent states
  24. Chapman-Kolmogorov equation
  25. Markov chains: accessibility and state classes
  26. Markov chains: long run proportion is inverse of time to reenter (incomplete)
  27. Markov chains: positive recurrence
  28. Markov chains: positive recurrence is a class property