GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → microsoft → Bringing Old Photos Back To Life

microsoft / Bringing Old Photos Back To Life

Licence: mit
Bringing Old Photo Back to Life (CVPR 2020 oral)

Programming Languages

python
139335 projects - #7 most used programming language

Labels

pytorchgenerative-adversarial-networkphotosimage-manipulationgansimage-restorationold-photo-restorationphoto-restoration

Projects that are alternatives of or similar to Bringing Old Photos Back To Life

Contrastive Unpaired Translation
Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)
Stars: ✭ 822 (-91.37%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Anycost Gan
[CVPR 2021] Anycost GANs for Interactive Image Synthesis and Editing
Stars: ✭ 367 (-96.15%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Faceswap Gan
A denoising autoencoder + adversarial losses and attention mechanisms for face swapping.
Stars: ✭ 3,099 (-67.46%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Cyclegan
Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
Stars: ✭ 10,933 (+14.78%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Cocosnet
Cross-domain Correspondence Learning for Exemplar-based Image Translation. (CVPR 2020 Oral)
Stars: ✭ 211 (-97.78%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Gesturegan
[ACM MM 2018 Oral] GestureGAN for Hand Gesture-to-Gesture Translation in the Wild
Stars: ✭ 136 (-98.57%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Selectiongan
[CVPR 2019 Oral] Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation
Stars: ✭ 366 (-96.16%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Pytorch Cyclegan And Pix2pix
Image-to-Image Translation in PyTorch
Stars: ✭ 16,477 (+72.99%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
Finegan
FineGAN: Unsupervised Hierarchical Disentanglement for Fine-grained Object Generation and Discovery
Stars: ✭ 240 (-97.48%)
Mutual labels:  generative-adversarial-network, gans, image-manipulation
UEGAN
[TIP2020] Pytorch implementation of "Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network"
Stars: ✭ 68 (-99.29%)
Mutual labels:  generative-adversarial-network, image-manipulation
Generative models tutorial with demo
Generative Models Tutorial with Demo: Bayesian Classifier Sampling, Variational Auto Encoder (VAE), Generative Adversial Networks (GANs), Popular GANs Architectures, Auto-Regressive Models, Important Generative Model Papers, Courses, etc..
Stars: ✭ 276 (-97.1%)
Mutual labels:  generative-adversarial-network, gans
Deep Generative Prior
Code for deep generative prior (ECCV2020 oral)
Stars: ✭ 308 (-96.77%)
Mutual labels:  generative-adversarial-network, image-manipulation
HistoGAN
Reference code for the paper HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms (CVPR 2021).
Stars: ✭ 158 (-98.34%)
Mutual labels:  image-manipulation, gans
VSGAN
VapourSynth Single Image Super-Resolution Generative Adversarial Network (GAN)
Stars: ✭ 124 (-98.7%)
Mutual labels:  image-manipulation, gans
DLSS
Deep Learning Super Sampling with Deep Convolutional Generative Adversarial Networks.
Stars: ✭ 88 (-99.08%)
Mutual labels:  generative-adversarial-network, gans
Deep-Learning
It contains the coursework and the practice I have done while learning Deep Learning.🚀 👨‍💻💥 🚩🌈
Stars: ✭ 21 (-99.78%)
Mutual labels:  generative-adversarial-network, gans
Sdv
Synthetic Data Generation for tabular, relational and time series data.
Stars: ✭ 360 (-96.22%)
Mutual labels:  generative-adversarial-network, gans
Igan
Interactive Image Generation via Generative Adversarial Networks
Stars: ✭ 3,845 (-59.63%)
Mutual labels:  generative-adversarial-network, image-manipulation
Data Efficient Gans
[NeurIPS 2020] Differentiable Augmentation for Data-Efficient GAN Training
Stars: ✭ 682 (-92.84%)
Mutual labels:  generative-adversarial-network, gans
Von
[NeurIPS 2018] Visual Object Networks: Image Generation with Disentangled 3D Representation.
Stars: ✭ 497 (-94.78%)
Mutual labels:  generative-adversarial-network, gans
View All Similar Projects ➔

Old Photo Restoration (Official PyTorch Implementation)

Project Page | Paper (CVPR version) | Paper (Journal version) | Pretrained Model | Colab Demo | Replicate Demo & Docker Image 🔥

Bringing Old Photos Back to Life, CVPR2020 (Oral)

Old Photo Restoration via Deep Latent Space Translation, PAMI Under Review

Ziyu Wan1, Bo Zhang2, Dongdong Chen3, Pan Zhang4, Dong Chen2, Jing Liao1, Fang Wen2
1City University of Hong Kong, 2Microsoft Research Asia, 3Microsoft Cloud AI, 4USTC

✨ News

The framework now supports the restoration of high-resolution input.

Training code is available and welcome to have a try and learn the training details.

You can now play with our Colab and try it on your photos.

Requirement

The code is tested on Ubuntu with Nvidia GPUs and CUDA installed. Python>=3.6 is required to run the code.

Installation

Clone the Synchronized-BatchNorm-PyTorch repository for

cd Face_Enhancement/models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../../

cd Global/detection_models
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../

Download the landmark detection pretrained model

cd Face_Detection/
wget http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
bzip2 -d shape_predictor_68_face_landmarks.dat.bz2
cd ../

Download the pretrained model from Azure Blob Storage, put the file Face_Enhancement/checkpoints.zip under ./Face_Enhancement, and put the file Global/checkpoints.zip under ./Global. Then unzip them respectively.

cd Face_Enhancement/
wget https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/face_checkpoints.zip
unzip face_checkpoints.zip
cd ../
cd Global/
wget https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/global_checkpoints.zip
unzip global_checkpoints.zip
cd ../

Install dependencies:

pip install -r requirements.txt

🚀 How to use?

Note: GPU can be set 0 or 0,1,2 or 0,2; use -1 for CPU

1) Full Pipeline

You could easily restore the old photos with one simple command after installation and downloading the pretrained model.

For images without scratches:

python run.py --input_folder [test_image_folder_path] \
              --output_folder [output_path] \
              --GPU 0

For scratched images:

python run.py --input_folder [test_image_folder_path] \
              --output_folder [output_path] \
              --GPU 0 \
              --with_scratch

For high-resolution images with scratches:

python run.py --input_folder [test_image_folder_path] \
              --output_folder [output_path] \
              --GPU 0 \
              --with_scratch \
              --HR

Note: Please try to use the absolute path. The final results will be saved in ./output_path/final_output/. You could also check the produced results of different steps in output_path.

2) Scratch Detection

Currently we don't plan to release the scratched old photos dataset with labels directly. If you want to get the paired data, you could use our pretrained model to test the collected images to obtain the labels.

cd Global/
python detection.py --test_path [test_image_folder_path] \
                    --output_dir [output_path] \
                    --input_size [resize_256|full_size|scale_256]

3) Global Restoration

A triplet domain translation network is proposed to solve both structured degradation and unstructured degradation of old photos.

cd Global/
python test.py --Scratch_and_Quality_restore \
               --test_input [test_image_folder_path] \
               --test_mask [corresponding mask] \
               --outputs_dir [output_path]

python test.py --Quality_restore \
               --test_input [test_image_folder_path] \
               --outputs_dir [output_path]

4) Face Enhancement

We use a progressive generator to refine the face regions of old photos. More details could be found in our journal submission and ./Face_Enhancement folder.

NOTE: This repo is mainly for research purpose and we have not yet optimized the running performance.

Since the model is pretrained with 256*256 images, the model may not work ideally for arbitrary resolution.

5) GUI

A user-friendly GUI which takes input of image by user and shows result in respective window.

How it works:

  1. Run GUI.py file.
  2. Click browse and select your image from test_images/old_w_scratch folder to remove scratches.
  3. Click Modify Photo button.
  4. Wait for a while and see results on GUI window.
  5. Exit window by clicking Exit Window and get your result image in output folder.

How to train?

1) Create Training File

Put the folders of VOC dataset, collected old photos (e.g., Real_L_old and Real_RGB_old) into one shared folder. Then

cd Global/data/
python Create_Bigfile.py

Note: Remember to modify the code based on your own environment.

2) Train the VAEs of domain A and domain B respectively

cd ..
python train_domain_A.py --use_v2_degradation --continue_train --training_dataset domain_A --name domainA_SR_old_photos --label_nc 0 --loadSize 256 --fineSize 256 --dataroot [your_data_folder] --no_instance --resize_or_crop crop_only --batchSize 100 --no_html --gpu_ids 0,1,2,3 --self_gen --nThreads 4 --n_downsample_global 3 --k_size 4 --use_v2 --mc 64 --start_r 1 --kl 1 --no_cgan --outputs_dir [your_output_folder] --checkpoints_dir [your_ckpt_folder]

python train_domain_B.py --continue_train --training_dataset domain_B --name domainB_old_photos --label_nc 0 --loadSize 256 --fineSize 256 --dataroot [your_data_folder]  --no_instance --resize_or_crop crop_only --batchSize 120 --no_html --gpu_ids 0,1,2,3 --self_gen --nThreads 4 --n_downsample_global 3 --k_size 4 --use_v2 --mc 64 --start_r 1 --kl 1 --no_cgan --outputs_dir [your_output_folder]  --checkpoints_dir [your_ckpt_folder]

Note: For the --name option, please ensure your experiment name contains "domainA" or "domainB", which will be used to select different dataset.

3) Train the mapping network between domains

Train the mapping without scratches:

python train_mapping.py --use_v2_degradation --training_dataset mapping --use_vae_which_epoch 200 --continue_train --name mapping_quality --label_nc 0 --loadSize 256 --fineSize 256 --dataroot [your_data_folder] --no_instance --resize_or_crop crop_only --batchSize 80 --no_html --gpu_ids 0,1,2,3 --nThreads 8 --load_pretrainA [ckpt_of_domainA_SR_old_photos] --load_pretrainB [ckpt_of_domainB_old_photos] --l2_feat 60 --n_downsample_global 3 --mc 64 --k_size 4 --start_r 1 --mapping_n_block 6 --map_mc 512 --use_l1_feat --niter 150 --niter_decay 100 --outputs_dir [your_output_folder] --checkpoints_dir [your_ckpt_folder]

Traing the mapping with scraches:

python train_mapping.py --no_TTUR --NL_res --random_hole --use_SN --correlation_renormalize --training_dataset mapping --NL_use_mask --NL_fusion_method combine --non_local Setting_42 --use_v2_degradation --use_vae_which_epoch 200 --continue_train --name mapping_scratch --label_nc 0 --loadSize 256 --fineSize 256 --dataroot [your_data_folder] --no_instance --resize_or_crop crop_only --batchSize 36 --no_html --gpu_ids 0,1,2,3 --nThreads 8 --load_pretrainA [ckpt_of_domainA_SR_old_photos] --load_pretrainB [ckpt_of_domainB_old_photos] --l2_feat 60 --n_downsample_global 3 --mc 64 --k_size 4 --start_r 1 --mapping_n_block 6 --map_mc 512 --use_l1_feat --niter 150 --niter_decay 100 --outputs_dir [your_output_folder] --checkpoints_dir [your_ckpt_folder] --irregular_mask [absolute_path_of_mask_file]

Traing the mapping with scraches (Multi-Scale Patch Attention for HR input):

python train_mapping.py --no_TTUR --NL_res --random_hole --use_SN --correlation_renormalize --training_dataset mapping --NL_use_mask --NL_fusion_method combine --non_local Setting_42 --use_v2_degradation --use_vae_which_epoch 200 --continue_train --name mapping_Patch_Attention --label_nc 0 --loadSize 256 --fineSize 256 --dataroot [your_data_folder] --no_instance --resize_or_crop crop_only --batchSize 36 --no_html --gpu_ids 0,1,2,3 --nThreads 8 --load_pretrainA [ckpt_of_domainA_SR_old_photos] --load_pretrainB [ckpt_of_domainB_old_photos] --l2_feat 60 --n_downsample_global 3 --mc 64 --k_size 4 --start_r 1 --mapping_n_block 6 --map_mc 512 --use_l1_feat --niter 150 --niter_decay 100 --outputs_dir [your_output_folder] --checkpoints_dir [your_ckpt_folder] --irregular_mask [absolute_path_of_mask_file] --mapping_exp 1

Citation

If you find our work useful for your research, please consider citing the following papers :)

@inproceedings{wan2020bringing,
title={Bringing Old Photos Back to Life},
author={Wan, Ziyu and Zhang, Bo and Chen, Dongdong and Zhang, Pan and Chen, Dong and Liao, Jing and Wen, Fang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={2747--2757},
year={2020}
}
@article{wan2020old,
  title={Old Photo Restoration via Deep Latent Space Translation},
  author={Wan, Ziyu and Zhang, Bo and Chen, Dongdong and Zhang, Pan and Chen, Dong and Liao, Jing and Wen, Fang},
  journal={arXiv preprint arXiv:2009.07047},
  year={2020}
}

If you are also interested in the legacy photo/video colorization, please refer to this work.

Maintenance

This project is currently maintained by Ziyu Wan and is for academic research use only. If you have any questions, feel free to contact [email protected].

License

The codes and the pretrained model in this repository are under the MIT license as specified by the LICENSE file. We use our labeled dataset to train the scratch detection model.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].