VQ-VAE with PixelCNN prior

Workflow

• Train the Vector Quantised Variational AutoEncoder (VQ-VAE) for discrete representation and reconstruction.
• Use PixelCNN to learn the priors on the discrete latents for image sampling.

Acknowledgement

• VQ-VAE is originally mentioned in the paper Neural Discrete Representation Learning.
• PixelCNN is proposed in the papers Pixel Recurrent Neural Networks and Conditional Image Generation with PixelCNN Decoders.
• Implementation of VQ-VAE (without priors) is based on the official codes from Google DeepMind. Note: Different from the official codes, the implementation here does not rely on the Sonnet library.
• Implementation of PixelCNN is based on this repo with little modify.
• We provide the slides which may be of help for readers to gain better understanding on PixelCNN and VQ-VAE. Some images used in the slides are borrowed from papers and websites, so the slides can only be used for learning purpose.

Usage

• Run MNIST: vqvae1_withPixelCNNprior_mnist.py
• Run cifar-10: vqvae1_withPixelCNNprior_cifar10.py

Results

	Testing data	Reconstruction	Random samples	Samples based on PixelCNN prior
MNIST
cifar-10