Learning Flow-based Feature Warping For Face Frontalization with Illumination Inconsistent Supervision
The source code for our paper "Learning Flow-based Feature Warping For Face Frontalization with Illumination Inconsistent Supervision" (ECCV 2020)
Quick Start
Installation
Prerequisites
- python3.7
- pytorch1.5.0 + torchvision0.6.0
- CUDA
- opencv-python
- numpy
- tensorboardX
- tqdm
Conda installation
# 1. Create a conda virtual environment.
conda create -n ffwm python=3.7 anaconda
source activate ffwm
# 2. Install the pytorch
conda install pytorch==1.5.0 torchvision==0.6.0 cudatoolkit=9.2 -c pytorch
# 3. Install dependency
pip install -r requirements.txt
# 4. Build pytorch Custom CUDA Extensions, we have tested it on pytorch1.5.0+cu92
bash setup.sh
Data Preparation
You can use the scripts in data_process to prepare your data.
The data folder is structured as follows:
dataset
├── multipie
│ ├── train
│ │ ├── images
│ │ ├── masks
│ │ └── landmarks.npy
│ └── test
│ ├── images
│ ├── gallery_list.npy (optional)
│ └── visual_list.npy (optional)
└── lfw
├── images
└── pairs.txt
Our test gallery_list.npy and visual_list.npy can download from GoogleDrive or BaiduNetDisk (l98p).
Testing
Download the models from GoogleDrive or BaiduNetDisk (l98p) to ./checkpoints folder or use your pretrained models. The models are structured as follows:
./checkpoints
├── ffwm
│ ├── latest_net_flowNetF.pth
│ └── latest_net_netG.pth
├── lightCNN_10_checkpoint.pth (pretrained)
└── LightCNN_29Layers_checkpoint.pth (original)
Test on MultiPIE
python test_ffwm.py \
--dataroot path/to/dataset \
--lightcnn path/to/pretrained lightcnn \
--preload
Test on LFW
python test_ffwm.py \
--datamode lfw \
--dataroot path/to/dataset \
--lightcnn path/to/pretrained lightcnn \
--preload
Training
1. Finetune LightCNN
cd lightcnn
python finetune.py \
--save_path ../checkpoints/ \
--dataroot path/to/dataset/multipie \
--model_path path/to/original lightcnn \
--preload
You can download the original LightCNN model from LightCNN. Or you can download the original and our pretrained LightCNN from GoogleDrive or BaiduNetDisk (l98p).
2. Train Forward FlowNet
python train_flow.py \
--model flownet \
--dataroot path/to/dataset \
--aug \
--preload \
--name flownetf \
--batch_size 6
3. Train Reverse FlowNet
python train_flow.py \
--model flownet \
--reverse \
--dataroot path/to/dataset \
--aug \
--preload \
--name flownetb \
--batch_size 6
4. Train FFWM
python train_ffwm.py \
--name ffwm \
--preload \
--dataroot path/to/dataset \
--lightcnn path/to/pretrained lightcnn
Citation
If you find our work useful in your research or publication, please cite:
@InProceedings{wei2020ffwm,
author = {Wei, Yuxiang and Liu, Ming and Wang, Haolin and Zhu, Ruifeng and Hu, Guosheng and Zuo, Wangmeng},
title = {Learning Flow-based Feature Warping For Face Frontalization with Illumination Inconsistent Supervision},
booktitle = {Proceedings of the European Conference on Computer Vision},
year = {2020}
}