VAE and Super-Resolution VAE in PyTorch
Code release for Super-Resolution Variational Auto-Encoders
Abstract
The framework of Variational Auto-Encoders (VAEs) provides a principled manner of reasoning in latent-variable models using variational inference. However, the main drawback of this approach is blurriness of generated images. Some studies link this effect to the objective function, namely, the (negative) log-likelihood function. Here, we propose to enhance VAEs by adding a random variable that is a downscaled version of the original image and still use the log-likelihood function as the learning objective. Further, we provide the downscaled image as an input to the decoder and use it in a manner similar to the super-resolution. We present empirically that the proposed approach performs comparably to VAEs in terms of the negative log-likelihood function, but it obtains a better FID score.
Features
-
Models
- VAE
- Super-resolution VAE (srVAE)
-
Priors
- Standard (unimodal) Gaussian
- Mixture of Gaussians
- RealNVP
-
Reconstruction Loss
- Discretized Mixture of Logistics Loss
-
Neural Networks
- DenseNet
-
Datasets
- CIFAR-10
Quantitative results
| Model | nll |
|---|---|
| VAE | 3.51 |
| srVAE | 3.65 |
Results on CIFAR-10. The log-likelihood value nll was estimated using 500 weighted samples on the test set (10k images).
Qualitative results
VAE
Results from VAE with RealNVP Prior trained on CIFAR10.
Interpolations
Reconstructions.
Unconditional generations.
Super-Resolution VAE
Results from Super-Resolution VAE trained on CIFAR10.
Interpolations
Super-Resolution results of the srVAE on CIFAR-10
Unconditional generations. Left: The generations of the first step, the compressed representations that capture the _global_ structure. Right: The final result after enhasing the images with local content.
Requirements
The code is compatible with:
python 3.6pytorch 1.3
Usage
- To run VAE with RealNVP prior on CIFAR-10, please execude:
python main.py --model VAE --network densenet32 --prior RealNVP
- Otherwise, to run srVAE:
python main.py --model srVAE --network densenet16x32 --prior RealNVP
Cite
Please cite our paper if you use this code in your own work:
@misc{gatopoulos2020superresolution,
title={Super-resolution Variational Auto-Encoders},
author={Ioannis Gatopoulos and Maarten Stol and Jakub M. Tomczak},
year={2020},
eprint={2006.05218},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Acknowledgements
This work was supported and funded from the University of Amsterdam, and BrainCreators B.V..
Repo Author
Ioannis Gatopoulos, 2020