Markov chains: finite sink is recurrent (incomplete)

Dependencies: (incomplete)

  1. Markov chain
  2. Markov chains: recurrent states
  3. Markov chains: accessibility and state classes
  4. Markov chains: recurrence is a class property
  5. Markov chains: recurrent iff expected number of visits is infinite

Let $X = [X_0, X_1, \ldots]$ be a markov chain having state space $D$. Let $I$ be a state class containing a finite number of states. Then $I$ is recurrent if no other state class is accessible from $I$.

Proof

For any state $i$, let $N_i$ be the number of times the state becomes $i$. Formally, let $N_i = \sum_{t=0}^{\infty} \begin{cases}1&\textrm{ if }X_t=i \\0&\textrm{ otherwise}\end{cases}$.

Let $I = \{i_1, i_2, \ldots, i_r\}$. Let $j \not\in I$. Then $j$ is not accessible from $i_1$, so $\Pr(N_j = 0 \mid X_0 = i_1) = 1$. Hence, $\E(N_j \mid X_0 = i_1) = 0$. However, $\sum_{i \in D} N_i = \infty$, so $\E(N_{i_k} \mid X_0 = i_1) = \infty$ for some $k$. (incomplete proof)

Dependency for: None

Info:

Transitive dependencies:

  1. /analysis/topological-space
  2. /sets-and-relations/countable-set
  3. /sets-and-relations/poset
  4. /sets-and-relations/equivalence-relation
  5. /sets-and-relations/de-morgan-laws
  6. /measure-theory/linearity-of-lebesgue-integral
  7. /measure-theory/lebesgue-integral
  8. σ-algebra
  9. Generated σ-algebra
  10. Borel algebra
  11. Measurable function
  12. Generators of the real Borel algebra (incomplete)
  13. Measure
  14. σ-algebra is closed under countable intersections
  15. Group
  16. Ring
  17. Field
  18. Vector Space
  19. Semiring
  20. Matrix
  21. Identity matrix
  22. Probability
  23. Conditional probability (incomplete)
  24. Random variable
  25. Markov chain
  26. Markov chains: recurrent states
  27. Chapman-Kolmogorov equation
  28. Markov chains: accessibility and state classes
  29. Expected value of a random variable
  30. Linearity of expectation
  31. Markov chains: recurrent iff expected number of visits is infinite
  32. Markov chains: recurrence is a class property