Caffe with GuidedNet
This the relase of the CVPR 2017 BNMW workshop version of GuidedNet, "Guided Optical Flow Learning". You can refer to paper for more details at Openreview or Arxiv.
The code base is heavily borrowed from DispNet and UnsupFlownet. Thanks for open sourcing the code.
Dependencies
OpenCV 3 (Installation can be refered here)
Tested on Ubuntu 16.04 with Titan X GPU, CUDNN 5.1
Compiling
To get started with GuidedNet, first compile caffe, by configuring a
"Makefile.config"
then make with
$ make -j 6 all
Training
(this assumes you compiled the code sucessfully)
First you need to download and prepare the training data. For that go to the data folder:
cd data
Then run:
./download.sh
Then prepare FlyingChairs_release.list file, change the directory accordingly. For example, if you want to train with FlowFields proxy ground truth, you need to first generate FlowFields flow estimation by yourself and change:
FlyingChairs_release/data/00001_img1.ppm FlyingChairs_release/data/00001_img2.ppm FlyingChairs_release/data/00001_flow.flo
to:
FlyingChairs_release/data/00001_img1.ppm FlyingChairs_release/data/00001_img2.ppm FlyingChairs_release/FlowFields/00001_flow.flo
Then run:
./make-lmdbs.sh
(this will take some time and disk space)
To train a GuidedNet network, go to this folder:
cd ./GuidedNet/models/FlowNetS_FlowFields/
Then just run:
./train.py
To train GuidedNet network with unsuervised fine-tuning, go to this folder:
cd ./GuidedNet/models/Unsup_FineTune/
Then just run:
./train.py
NOTE: You may get better performance if you carefully tune the hyper-params for different datasets, such as loss weights, learning rate etc.
Testing
(this assumes you compiled the code sucessfully)
E.g. go to this folder:
cd ./GuidedNet/models/FlowNetS_FlowFields/
First, download pre-trained models:
FlowNetS trained with FlowFields proxy ground truth for Flying Chairs dataset
FlowNetS trained with FlowFields and unsupervised fine-tuning for Flying Chairs dataset
FlowNetS trained with FlowFields and unsupervised fine-tuning for MPI-Sintel dataset
Then prepare image pairs list, as in the example of fc_val_im0.txt and fc_val_im1.txt.
Change line 157 in run.py to use the correct model. To try out GuidedNet on sample image pairs, run
./run.py fc_val_im0.txt fc_val_im1.txt
This is just an example of testing on Flying Chairs dataset. You can generate your own image pairs list for any dataset.
License and Citation
Please cite this paper in your publications if you use GuidedNet for your research:
@article{guided_flow_17,
title={{Guided Optical Flow Learning}},
author={Yi Zhu and Zhenzhong Lan and Shawn Newsam and Alexander G. Hauptmann},
journal={arXiv preprint arXiv:1702.022952},
year={2017}
}