Standard normal random vector on vector space

Dependencies:

  1. Standard multivariate normal distribution
  2. Basis of a vector space
  3. Identity matrix
  4. Product of stacked matrices
  5. Orthogonality and orthonormality
  6. Gram-Schmidt Process
  7. Orthonormal basis change matrix

Let $\mathcal{V} \subseteq \mathbb{R}^n$ be an inner-product space with inner-product defined as $\langle x, y \rangle = x^Ty$. Suppose $\dim(\mathcal{V}) = m$.

Let $U = [u_1, u_2, \ldots, u_m]$ be an orthonormal basis of $\mathcal{V}$. Such a basis is guaranteed to exist by the Gram-Schmidt process. Interpret $U$ as a $n$-by-$m$ matrix whose $i^{\textrm{th}}$ column is $u_i$.

Let $Z = [Z_1, Z_2, \ldots, Z_m] \sim \mathcal{N}(0, I)$ be a random vector. Let $X = \sum_{i=1}^m Z_i u_i$. Then $X$ is called the standard normal vector on vector space $\mathcal{V}$.

$X = UZ \sim \mathcal{N}(0, UU^T)$ (by theorem on product of stacked matrices and multivariate normal distribution).

$UU^T$ is independent of the choice of $U$ (it depends only on $\mathcal{V}$), so $X$ is uniquely defined for $\mathcal{V}$.

Proof of $UU^T$ being independent of the choice of $U$

Let $V = [v_1, v_2, \ldots, v_m]$ be another orthonormal basis of $\mathcal{V}$. Interpret $V$ as a $n$-by-$m$ matrix whose $i^{\textrm{th}}$ column is $v_i$.

Let $A$ be the basis-change matrix from $V$ to $U$, i.e. \[ v_i = \sum_{j=1}^m A[i, j]u_j \] Then $A$ is an orthogonal $m$-by-$m$ matrix, so $AA^T = A^TA = I$.

\[ V[i, j] = (v_j)_i = \sum_{k=1}^n A[j, k](u_k)_i = \sum_{k=1}^n A[j, k]U[i, k] = (UA^T)[i, j] \] Therefore, $V = UA^T$.

\[ VV^T = (UA^T)(UA^T)^T = UA^TAU^T = UU^T \] Therefore, every orthonormal basis $V$ has the same value of $VV^T$.

Dependency for: None

Info:

Transitive dependencies:

  1. /analysis/exponential-grows-superpolynomially
  2. /analysis/integration-by-parts
  3. /measure-theory/linearity-of-lebesgue-integral
  4. /measure-theory/lebesgue-integral
  5. /linear-algebra/matrices/gauss-jordan-algo
  6. /linear-algebra/vector-spaces/condition-for-subspace
  7. /complex-numbers/conjugate-product-abs
  8. /complex-numbers/conjugation-is-homomorphic
  9. /sets-and-relations/equivalence-relation
  10. /sets-and-relations/de-morgan-laws
  11. /sets-and-relations/countable-set
  12. /analysis/topological-space
  13. Gaussian integral (incomplete)
  14. Group
  15. Ring
  16. Polynomial
  17. Vector
  18. Dot-product of vectors
  19. Field
  20. Vector Space
  21. Inner product space
  22. Orthogonality and orthonormality
  23. Linear independence
  24. Span
  25. Gram-Schmidt Process
  26. Integral Domain
  27. Comparing coefficients of a polynomial with disjoint variables
  28. Semiring
  29. Matrix
  30. Stacking
  31. System of linear equations
  32. Transpose of stacked matrix
  33. Product of stacked matrices
  34. Matrix multiplication is associative
  35. Trace of a matrix
  36. Transpose of product
  37. Reduced Row Echelon Form (RREF)
  38. Elementary row operation
  39. Every elementary row operation has a unique inverse
  40. Row equivalence of matrices
  41. Matrices over a field form a vector space
  42. Row space
  43. Row equivalent matrices have the same row space
  44. RREF is unique
  45. Matrices form an inner-product space
  46. Identity matrix
  47. Full-rank square matrix in RREF is the identity matrix
  48. Inverse of a matrix
  49. Inverse of product
  50. Elementary row operation is matrix pre-multiplication
  51. Row equivalence matrix
  52. Equations with row equivalent matrices have the same solution set
  53. Basis of a vector space
  54. Linearly independent set is not bigger than a span
  55. Homogeneous linear equations with more variables than equations
  56. Rank of a homogenous system of linear equations
  57. Rank of a matrix
  58. Full-rank square matrix is invertible
  59. AB = I implies BA = I
  60. Orthogonal matrix
  61. Orthonormal basis change matrix
  62. σ-algebra
  63. σ-algebra is closed under countable intersections
  64. Measure
  65. Probability
  66. Conditional probability (incomplete)
  67. Independence of events
  68. Independence of composite events
  69. Generated σ-algebra
  70. Measurable function
  71. Borel algebra
  72. Generators of the real Borel algebra (incomplete)
  73. Random variable
  74. Expected value of a random variable
  75. Linearity of expectation
  76. Covariance of 2 random variables
  77. Variance of a random variable
  78. Normal distribution
  79. Linearity of expectation for matrices
  80. Cross-covariance matrix
  81. Covariance matrix
  82. Independence of random variables (incomplete)
  83. Standard multivariate normal distribution