VPN: Video Pixel Networks in Tensorflow
The VPN is a probabilistic video model that estimates the discrete joint distribution of the raw pixel values in a video. It approaches the best possible performance on the Moving MNIST benchmark.
This repository contains a tensorflow implementation of the VPN architecture proposed in the paper. However, this code hasn’t been trained and tested on the full Moving MNIST dataset due to lack of the computation power. It has been overfitted on one sequence to insure the correctness of the implementation.
This repository also contains some additional experiments with the VPN architecture that are not mentioned in the original paper. These experiments are:
- Mini VPN architecture.
- Micro VPN architecture.
Examples of the network output
Here's different steps from predicting the first frame pixel-by-pixel:
The loss curves for the different architectures
full:pink
mini:red
micro:orange
The Moving MNIST Dataset
You can download the full moving MNIST dataset from toronto website.
Overfitting On One Sequence
python vpn.py --vpn_arch='mini' --train=True --overfitting=Ture --data_dir='/numpy/file/directory/' --exp_dir='/tmp/vpn/'
Training On The Full Dataset
python vpn.py --vpn_arch='mini' --train=True --overfitting=False --data_dir='/numpy/file/directory/' --exp_dir='/tmp/vpn/'
TODO
- Train on data generated on the air.