Skip to content

Linear algebra


Matrix properties


If vector is in the span of a set of vectors , it means that it can be recreated by a linear combination of those vectors.

Singular matrix

  • A matrix which removes some of the dimensions as part of its transformation.
  • Has a null space (different inputs produce the same output).

Positive definite matrix

Positive semidefinite matrix

Real symmetric matrix

Hermitian matrix

  • Equal to it's own conjugate transform.
  • Complex positive (semi)definite matrices are Hermitian.
  • Always has real eigenvalues (just like real symmetric matrices).

Gramm matrix

A matrix where every element is a dot product in some feature space.

Hilbert space

  • Euclidean space generalized to infinite dimensions
  • Can contain functions, since a function is pretty much the same as a vector of infinite length.
  • Complete, meaning there are no "holes". For example, rational numbers are not complete because is not in it. Complete means you can work with limits of sequences.
  • Vectors in Hilbert space have a concept of length and inner product.
  • Consider the set of all sine waves which fit an integer amount of times on a string. You can create an arbitrary function by combining these together with appropriate coefficients. So we can consider each sine wave to each be an independent dimension in a Hilbert space. The function is now a point in this abstract space.

Reproducing kernel Hilbert space

A Reproducing Kernel Hilbert Space (RKHS) is a Hilbert space H with a reproducing kernel whose span is dense in . We could equivalently define an RKHS as a Hilbert space of functions with all evaluation functionals bounded and linear.

Matrix transformations

Conjugate transform

Take a transform and a complex-conjugate of imaginary numbers.

Direct sum

Tensor product

  • Mapping which takes two inputs and produces a real number.
  • Tensor is a map which takes elements from the Cartesian space of vectors and produces a real number.
  • Tensorts come in many forms because there are many different ways of making Cartesian products?


  • Represent a matrix in terms of its eigenvalues and eigenvectors.
  • Real symmetric matrices have real eigenvalues and orthogonal eigenvectors.
  • Singular value decomposition extends this to non-square matrices.
  • Wikipedia