4PP-EUSR
Four-pass perceptual super-resolution with enhanced upscaling
Introduction
This repository contains a TensorFlow-based implementation of 4PP-EUSR ("Deep learning-based image super-resolution considering quantitative and perceptual quality"), which considers both the quantitative (e.g., PSNR) and perceptual quality (e.g., NIQE) of the upscaled images. Our method won the 2nd place and got the highest human opinion score for Region 2 in the 2018 PIRM Challenge on Perceptual Image Super-resolution at ECCV 2018.
※ The perceptual index is calculated by "0.5 * ((10 - Ma) + NIQE)", which is used in the PIRM Challenge. Lower is better.
Followings are the performance comparison evaluated on the BSD100 dataset.
| Method | PSNR (dB) (↓) | Perceptual Index |
|---|---|---|
| EDSR | 27.796 | 5.326 |
| MDSR | 27.771 | 5.424 |
| EUSR | 27.674 | 5.307 |
| SRResNet-MSE | 27.601 | 5.217 |
| 4PP-EUSR (PIRM Challenge) | 26.569 | 2.683 |
| SRResNet-VGG22 | 26.322 | 5.183 |
| SRGAN-MSE | 25.981 | 2.802 |
| Bicubic interpolation | 25.957 | 6.995 |
| SRGAN-VGG22 | 25.697 | 2.631 |
| SRGAN-VGG54 | 25.176 | 2.351 |
| CX | 24.581 | 2.250 |
Please cite following papers when you use the code, pre-trained models, or results:
- J.-H. Choi, J.-H. Kim, M. Cheon, J.-S. Lee: Deep learning-based image super-resolution considering quantitative and perceptual quality. Neurocomputing (In Press) [Paper] [arXiv]
@article{choi2018deep,
title={Deep learning-based image super-resolution considering quantitative and perceptual quality},
author={Choi, Jun-Ho and Kim, Jun-Hyuk and Cheon, Manri and Lee, Jong-Seok},
journal={Neurocomputing},
year={2019},
publisher={Elsevier}
}
- J.-H. Kim, J.-S. Lee: Deep residual network with enhanced upscaling module for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 913-921 (2018) [Paper]
@inproceedings{kim2018deep,
title={Deep residual network with enhanced upscaling module for super-resolution},
author={Kim, Jun-Hyuk and Lee, Jong-Seok},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
year={2018}
}
Dependencies
- Python 3.6+
- TensorFlow 1.11+ (<2.0)
Testing pre-trained models
※ PIRM Challenge version is available on SRZoo.
Generating upscaled images from the trained models can be done by test/test.py.
Following are the brief instructions.
- Download and copy the trained model available in Downloads section to the
test/folder. - Place the low-resolution images (PNG only) to the
test/LR/folder. - Run
python test.py --model_name [model file name]. For example, if you downloaded the PIRM Challenge version of our pre-trained model, runpython test.py --model_name 4pp_eusr_pirm.pb. - The upscaled images will be available on the
test/SR/folder.
Please run python test.py --help for more information.
Training EUSR, score predictors, and 4PP-EUSR
Please refer to README.md on the train/ folder for more information.
Downloads
Pre-trained models:
- PIRM Challenge version: 4pp_eusr_pirm.pb
- Paper version (v2): 4pp_eusr_paper_v2.pb
Results (Set5, Set14, BSD100, PIRM):
- PIRM Challenge version: 4pp_eusr_results_pirm.zip
- Paper version (v2): 4pp_eusr_results_paper_v2.zip