SeRanet

Super Resolution of picture images using deep learning.

Demo

SeRanet upscales picture image size to x2.

* Below sample pictures are obtained from PEXELS, providing public domain photos.

Before upscale

Conventional method: image upscaled with lanczos method (with OpenCV library)

SeRanet: image upscaled using seranet_v1

Original image (= Ground truth data, for reference)

Description

SeRanet project aims to Study and Research how deep convolutional neural network works to learn super resolution of the image.

It is developed with python on chainer framework, flexible machine learning library.

Explanation material about super resolution through convolutional neural network is uploaded on SlideShare.

• SeRanet introduction - SlideShare
• SeRanet introduction - SlideShare (Japanese)

References

The project is inspired by following two reference

• Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang, "Image Super-Resolution Using Deep Convolutional Networks", http://arxiv.org/abs/1501.00092

The term "SRCNN", super resolution using deep convolutional neural network, is introduced in this paper.

• waifu2x

Image super resolution for Anime-Style art and photo. You can also test the application at http://waifu2x.udp.jp/index.html.

Machine learning library

• chainer

Machine learning library which can be written in python. It is open source on github

Environment setup and requirements

I tested the software with Ubuntu 14.04, python 2.7.

Running the software requires a lot of memory, more than 8GB RAM recommended (it depends on which architecture to use).

• numpy

• chainer

Follow github page for installation. Setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is recommended to use GPU if possible.

• OpenCV

OpenCV is used for image processing. Installation page

Also, please set environment variable PYTHONPATH='.'. Add below code in ~/.bashrc.

export PYTHONPATH=.:${PYTHONPATH}

How to use

Basic usage

Just specify image file path which you want to upscale. If you get memory error or exit code 137, please try with smaller size image.

Ex. Upscaling input.jpg

python src/inference.py input.jpg

Specify output file name and path

Ex. Upscaling /path/to/input.jpg to /path/to/output.jpg

python src/inference.py /path/to/input.jpg /path/to/output.jpg

Specify model to use:

SeRanet project studies several network architecture. You can specify which network archtecture to use for SR with -a option.

Ex. use model seranet_v1

python src/inference.py /path/to/input.jpg /path/to/output.jpg -a seranet_v1

Use GPU:

GPU makes calculation much faster. Specify -g option is to use GPU.

python src/inference.py /path/to/input.jpg /path/to/output.jpg -g 0

Training

You can construct your own convolutional neural network, and train it!

1. Data preparation

Put training images[1] inside data/training_images directory. (I used 5000 photo images during the training, see next section "Training images".)

[1]: Currently, images will be cropped to size 232 x 232 during training.

2. Training the model

Once prepared training_images, see code/arch/ directory to choose which model to train, and execute below. -g 0 is to use GPU. (For the training, it is highly recommended to use GPU, otherwise training don't finish maybe a month...)

python src/train.py -a model_name -g 0

Training images

Training images may be more important than algorithm of deep learning to achieve high quality super resolution. Generally, training images dataset distributed for image classification task are not so high quality in terms of sharpness of the images.

Therefore, I chose PEXELS photos for the training of SeRanet.

• PEXELS

The website provides high quality photos under Creative Commons Zero (CC0) license.

Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,

• Download PEXELS training_images (jump to google drive, file size around 269MB)

After extract, copy this to data/training_images to start your own training.

Contribution is welcome

The performance of SR for this project is still not matured. You are welcome to suggest any improvement & contribute to this project. If you could get any model which performs better performance, feel free to send me a pull request!