Modeling Continuous Stochastic Process with Dynamic Normalizing Flow
Code for the paper
Ruizhi Deng, Bo Chang, Marcus Brubaker, Greg Mori, Andreas Lehrmann. "Modeling Continuous Stochastic Process with Dynamic Normalizing Flow" (2020) [arxiv]
Dependency installment
Install the dependencies in requirements.txt with
pip install -r requirements.txt -f https://download.pytorch.org/whl/torch_stable.htmlWe use a new version of PyTorch (1.4.0) for the released code. The experiments in the paper use an older version of PyTorch, which could lead to slightly different results. Please read the PyTorch documentation for more information.
Acknowledgements
The code make uses of code from the following two projects: https://github.com/YuliaRubanova/latent_ode for the paper
Yulia Rubanova, Ricky Chen, David Duvenaud. "Latent ODEs for Irregularly-Sampled Time Series" (2019) [arxiv]
https://github.com/rtqichen/ffjord for the paper
Will Grathwohl*, Ricky T. Q. Chen*, Jesse Bettencourt, Ilya Sutskever, David Duvenaud. "FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models." International Conference on Learning Representations (2019). [arxiv] [bibtex]
We make use the following files from the code of FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models: train_misc.py, lib/layers, lib/utils.py, lib/spectral_norm.py.
We make changes to the file train_misc.py, lib/layers/ode_func.py, and lib/utils.py.
We use lib/encoder_decoder.py, lib/ode_func.py, lib/diffeq_solver.py from the code of Latent ODEs for Irregularly-Sampled Time Series. We make changes to the file lib/encoder_decoder.py.
Command for training the model
We train the models using
Training CTFP model on GBM Process
python train_ctfp.py --batch_size 100 --test_batch_size 100 --num_blocks 1 --save ctfp_gbm --log_freq 1 --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 100 --data_path data/gbm_2.pklTraining latent CTFP model on GBM Process
python train_latent_ctfp.py --batch_size 50 --test_batch_size 5 --num_blocks 1 --save latent_ctfp_gbm --log_freq 1 --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 100 --data_path data/gbm_2.pklTraining CTFP model on OU Process
python train_ctfp.py --batch_size 100 --test_batch_size 100 --num_blocks 1 --save ctfp_ou --log_freq 1 --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 100 --data_path data/ou_2.pkl --activation identityTraining latent CTFP model on OU Process
python train_latent_ctfp.py --batch_size 50 --test_batch_size 5 --num_blocks 1 --save latent_ctfp_ou --log_freq 1 --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 300 --data_path data/ou_2.pkl --activation identity --aggressiveCommand for evaluating the code
We evaluate the models on data sampled by a observation process with
Evaluating CTFP model on GBM Process
python eval_ctfp.py --test_batch_size 100 --num_blocks 1 --save ctfp_gbm --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 100 --resume experiments/ctfp_gbm/pretrained.pth --data_path data/gbm_2.pklEvaluating latent CTFP model on GBM Process
python eval_latent_ctfp.py --test_batch_size 5 --num_blocks 1 --save latent_ctfp_gbm --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --lr 5e-4 --num_epochs 100 --data_path data/gbm_2.pkl --resume experiments/latent_ctfp_gbm/pretrained.pthEvaluating CTFP model on OU Process
python eval_ctfp.py --test_batch_size 100 --num_blocks 1 --save ctfp_ou --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --data_path data/ou_2.pkl --activation identity --resume experiments/ctfp_ou/pretrained.pthEvaluating latent CTFP model on OU Process
python eval_latent_ctfp.py --test_batch_size 5 --num_blocks 1 --save latent_ctfp_ou --num_workers 2 --layer_type concat --dims 32,64,64,32 --nonlinearity tanh --data_path data/ou_2.pkl --activation identity --resume experiments/latent_ctfp_ou/pretrained.pthPerformance Summary
| Model | GBM | OU |
|---|---|---|
| CTFP | 3.107 | 2.902 |
| Latent CTFP | 3.106 | 2.902 |
Download the data from this link for evaluating the models on GBM and OU data with