Variational Autoencoders

Like autoencoders, but penalize latent variables for deviating from a Gaussian distribution.

See also:

• Variational Autoencoders Explained

$$p(z|x) = \frac{p(x|z) p(z)}{p(x)}$$

$p(x|z)$ can be calculated using our decoder neural network, which generates images by taking samples from the latent space $x$.

$p(z)$ is our prior over the latent space $z$ (usually a unit Gaussian).

$p(x) = \int {p(z) p(x) dz}$ is difficult to calculate because it requires calculating $p(x)$ for all possible values of $z$.

$q(z|x)$ is the variational posterior, parameterized by a second set of parameters $\lambda$, which approximates the true posterior distribution $p(z|x)$.

Additional resources

• Tutorial - What is a variational autoencoder?.
• CrossValidated - What are variational autoencoders and to what learning tasks are they used?.
• CrossValidated - How does the reparameterization trick for VAEs work and why is it important?.