Markov chains: finite sink is recurrent (incomplete)

Dependencies: (incomplete)

  1. Markov chain
  2. Markov chains: recurrent states
  3. Markov chains: accessibility and state classes
  4. Markov chains: recurrence is a class property
  5. Markov chains: recurrent iff expected number of visits is infinite

Let $X = [X_0, X_1, \ldots]$ be a markov chain having state space $D$. Let $I$ be a state class containing a finite number of states. Then $I$ is recurrent if no other state class is accessible from $I$.

Proof

For any state $i$, let $N_i$ be the number of times the state becomes $i$. Formally, let $N_i = \sum_{t=0}^{\infty} \begin{cases}1&\textrm{ if }X_t=i \\0&\textrm{ otherwise}\end{cases}$.

Let $I = \{i_1, i_2, \ldots, i_r\}$. Let $j \not\in I$. Then $j$ is not accessible from $i_1$, so $\Pr(N_j = 0 \mid X_0 = i_1) = 1$. Hence, $\E(N_j \mid X_0 = i_1) = 0$. However, $\sum_{i \in D} N_i = \infty$, so $\E(N_{i_k} \mid X_0 = i_1) = \infty$ for some $k$. (incomplete proof)

Dependency for: None

Info:

Transitive dependencies:

  1. /sets-and-relations/poset
  2. /measure-theory/linearity-of-lebesgue-integral
  3. /measure-theory/lebesgue-integral
  4. /sets-and-relations/equivalence-relation
  5. /sets-and-relations/de-morgan-laws
  6. /sets-and-relations/countable-set
  7. /analysis/topological-space
  8. Group
  9. Ring
  10. Field
  11. Vector Space
  12. Semiring
  13. Matrix
  14. Identity matrix
  15. σ-algebra
  16. σ-algebra is closed under countable intersections
  17. Measure
  18. Probability
  19. Conditional probability (incomplete)
  20. Generated σ-algebra
  21. Measurable function
  22. Borel algebra
  23. Generators of the real Borel algebra (incomplete)
  24. Random variable
  25. Markov chain
  26. Markov chains: recurrent states
  27. Chapman-Kolmogorov equation
  28. Markov chains: accessibility and state classes
  29. Expected value of a random variable
  30. Linearity of expectation
  31. Markov chains: recurrent iff expected number of visits is infinite
  32. Markov chains: recurrence is a class property