Standard normal random vector on vector space

Dependencies:

  1. Standard multivariate normal distribution
  2. Basis of a vector space
  3. Identity matrix
  4. Product of stacked matrices
  5. Orthogonality and orthonormality
  6. Gram-Schmidt Process
  7. Orthonormal basis change matrix

Let $\mathcal{V} \subseteq \mathbb{R}^n$ be an inner-product space with inner-product defined as $\langle x, y \rangle = x^Ty$. Suppose $\dim(\mathcal{V}) = m$.

Let $U = [u_1, u_2, \ldots, u_m]$ be an orthonormal basis of $\mathcal{V}$. Such a basis is guaranteed to exist by the Gram-Schmidt process. Interpret $U$ as a $n$-by-$m$ matrix whose $i^{\textrm{th}}$ column is $u_i$.

Let $Z = [Z_1, Z_2, \ldots, Z_m] \sim \mathcal{N}(0, I)$ be a random vector. Let $X = \sum_{i=1}^m Z_i u_i$. Then $X$ is called the standard normal vector on vector space $\mathcal{V}$.

$X = UZ \sim \mathcal{N}(0, UU^T)$ (by theorem on product of stacked matrices and multivariate normal distribution).

$UU^T$ is independent of the choice of $U$ (it depends only on $\mathcal{V}$), so $X$ is uniquely defined for $\mathcal{V}$.

Proof of $UU^T$ being independent of the choice of $U$

Let $V = [v_1, v_2, \ldots, v_m]$ be another orthonormal basis of $\mathcal{V}$. Interpret $V$ as a $n$-by-$m$ matrix whose $i^{\textrm{th}}$ column is $v_i$.

Let $A$ be the basis-change matrix from $V$ to $U$, i.e. \[ v_i = \sum_{j=1}^m A[i, j]u_j \] Then $A$ is an orthogonal $m$-by-$m$ matrix, so $AA^T = A^TA = I$.

\[ V[i, j] = (v_j)_i = \sum_{k=1}^n A[j, k](u_k)_i = \sum_{k=1}^n A[j, k]U[i, k] = (UA^T)[i, j] \] Therefore, $V = UA^T$.

\[ VV^T = (UA^T)(UA^T)^T = UA^TAU^T = UU^T \] Therefore, every orthonormal basis $V$ has the same value of $VV^T$.

Dependency for: None

Info:

Transitive dependencies:

  1. /analysis/topological-space
  2. /sets-and-relations/countable-set
  3. /linear-algebra/vector-spaces/condition-for-subspace
  4. /linear-algebra/matrices/gauss-jordan-algo
  5. /complex-numbers/conjugate-product-abs
  6. /complex-numbers/conjugation-is-homomorphic
  7. /sets-and-relations/equivalence-relation
  8. /analysis/exponential-grows-superpolynomially
  9. /analysis/integration-by-parts
  10. /sets-and-relations/de-morgan-laws
  11. /measure-theory/linearity-of-lebesgue-integral
  12. /measure-theory/lebesgue-integral
  13. σ-algebra
  14. Generated σ-algebra
  15. Borel algebra
  16. Measurable function
  17. Generators of the real Borel algebra (incomplete)
  18. Measure
  19. σ-algebra is closed under countable intersections
  20. Group
  21. Ring
  22. Polynomial
  23. Vector
  24. Dot-product of vectors
  25. Integral Domain
  26. Comparing coefficients of a polynomial with disjoint variables
  27. Field
  28. Vector Space
  29. Linear independence
  30. Span
  31. Inner product space
  32. Orthogonality and orthonormality
  33. Gram-Schmidt Process
  34. Semiring
  35. Matrix
  36. Stacking
  37. System of linear equations
  38. Product of stacked matrices
  39. Transpose of stacked matrix
  40. Matrix multiplication is associative
  41. Reduced Row Echelon Form (RREF)
  42. Transpose of product
  43. Trace of a matrix
  44. Matrices over a field form a vector space
  45. Row space
  46. Matrices form an inner-product space
  47. Elementary row operation
  48. Every elementary row operation has a unique inverse
  49. Row equivalence of matrices
  50. Row equivalent matrices have the same row space
  51. RREF is unique
  52. Identity matrix
  53. Inverse of a matrix
  54. Inverse of product
  55. Elementary row operation is matrix pre-multiplication
  56. Row equivalence matrix
  57. Equations with row equivalent matrices have the same solution set
  58. Basis of a vector space
  59. Linearly independent set is not bigger than a span
  60. Homogeneous linear equations with more variables than equations
  61. Rank of a homogenous system of linear equations
  62. Rank of a matrix
  63. Full-rank square matrix in RREF is the identity matrix
  64. Full-rank square matrix is invertible
  65. AB = I implies BA = I
  66. Orthogonal matrix
  67. Orthonormal basis change matrix
  68. Probability
  69. Conditional probability (incomplete)
  70. Independence of events
  71. Independence of composite events
  72. Random variable
  73. Expected value of a random variable
  74. Independence of random variables (incomplete)
  75. Linearity of expectation
  76. Variance of a random variable
  77. Covariance of 2 random variables
  78. Linearity of expectation for matrices
  79. Cross-covariance matrix
  80. Covariance matrix
  81. Gaussian integral (incomplete)
  82. Normal distribution
  83. Standard multivariate normal distribution