# Linear algebra

## Matrix properties

#### Span

If vector $\mathbf{y}$ is in the span of a set of $m$ vectors $\mathbf{x}_1,...,\mathbf{x}_m$, it means that it can be recreated by a linear combination of those vectors.

#### Singular matrix

• A matrix which removes some of the dimensions as part of its transformation.
• Has a null space (different inputs produce the same output).

#### Hermitian matrix

• Equal to it's own conjugate transform.
• Complex positive (semi)definite matrices are Hermitian.
• Always has real eigenvalues (just like real symmetric matrices).

#### Gramm matrix

A matrix where every element is a dot product in some feature space.

#### Hilbert space

• Euclidean space generalized to infinite dimensions
• Can contain functions, since a function is pretty much the same as a vector of infinite length.
• Complete, meaning there are no "holes". For example, rational numbers are not complete because $\sqrt{2}$ is not in it. Complete means you can work with limits of sequences.
• Vectors in Hilbert space have a concept of length and inner product.
• Consider the set of all sine waves which fit an integer amount of times on a string. You can create an arbitrary function by combining these together with appropriate coefficients. So we can consider each sine wave to each be an independent dimension in a Hilbert space. The function is now a point in this abstract space.

#### Reproducing kernel Hilbert space

A Reproducing Kernel Hilbert Space (RKHS) is a Hilbert space H with a reproducing kernel whose span is dense in $\mathcal{H}$. We could equivalently define an RKHS as a Hilbert space of functions with all evaluation functionals bounded and linear.

## Matrix transformations

#### Conjugate transform

Take a transform and a complex-conjugate of imaginary numbers.

#### Tensor product

• Mapping which takes two inputs and produces a real number.
• Tensor is a map which takes elements from the Cartesian space of vectors and produces a real number.
• Tensorts come in many forms because there are many different ways of making Cartesian products?
• http://hitoshi.berkeley.edu/221a/tensorproduct.pdf.

#### Eigendecomposition

• Represent a matrix in terms of its eigenvalues and eigenvectors.
• Real symmetric matrices have real eigenvalues and orthogonal eigenvectors.
• Singular value decomposition extends this to non-square matrices.
• Wikipedia