Skip to content

Super Resolution of picture images using deep learning

License

Notifications You must be signed in to change notification settings

corochann/SeRanet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

36 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SeRanet

Super Resolution of picture images using deep learning.

Demo

SeRanet upscales picture image size to x2.

* Below sample pictures are obtained from PEXELS, providing public domain photos.

Before upscale

input picture1 input picture2 input picture3

Conventional method: image upscaled with lanczos method (with OpenCV library)

lanczos picture1 lanczos picture2 lanczos picture3

SeRanet: image upscaled using seranet_v1

seranet_v1 picture1 seranet_v1 picture2 seranet_v1 picture3

Original image (= Ground truth data, for reference)

original picture1 original picture2 original picture3

Description

SeRanet project aims to Study and Research how deep convolutional neural network works to learn super resolution of the image.

It is developed with python on chainer framework, flexible machine learning library.

Explanation material about super resolution through convolutional neural network is uploaded on SlideShare.

References

The project is inspired by following two reference

The term "SRCNN", super resolution using deep convolutional neural network, is introduced in this paper.

Image super resolution for Anime-Style art and photo. You can also test the application at http://waifu2x.udp.jp/index.html.

Machine learning library

Machine learning library which can be written in python. It is open source on github

Environment setup and requirements

I tested the software with Ubuntu 14.04, python 2.7.

Running the software requires a lot of memory, more than 8GB RAM recommended (it depends on which architecture to use).

  • numpy

  • chainer

Follow github page for installation. Setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is recommended to use GPU if possible.

  • OpenCV

OpenCV is used for image processing. Installation page

Also, please set environment variable PYTHONPATH='.'. Add below code in ~/.bashrc.

export PYTHONPATH=.:${PYTHONPATH}

How to use

Basic usage

Just specify image file path which you want to upscale. If you get memory error or exit code 137, please try with smaller size image.

Ex. Upscaling input.jpg

python src/inference.py input.jpg

Specify output file name and path

Ex. Upscaling /path/to/input.jpg to /path/to/output.jpg

python src/inference.py /path/to/input.jpg /path/to/output.jpg

Specify model to use:

SeRanet project studies several network architecture. You can specify which network archtecture to use for SR with -a option.

Ex. use model seranet_v1

python src/inference.py /path/to/input.jpg /path/to/output.jpg -a seranet_v1

Use GPU:

GPU makes calculation much faster. Specify -g option is to use GPU.

python src/inference.py /path/to/input.jpg /path/to/output.jpg -g 0

Training

You can construct your own convolutional neural network, and train it!

1. Data preparation

Put training images[1] inside data/training_images directory. (I used 5000 photo images during the training, see next section "Training images".)

[1]: Currently, images will be cropped to size 232 x 232 during training.

2. Training the model

Once prepared training_images, see code/arch/ directory to choose which model to train, and execute below. -g 0 is to use GPU. (For the training, it is highly recommended to use GPU, otherwise training don't finish maybe a month...)

python src/train.py -a model_name -g 0

Training images

Training images may be more important than algorithm of deep learning to achieve high quality super resolution. Generally, training images dataset distributed for image classification task are not so high quality in terms of sharpness of the images.

Therefore, I chose PEXELS photos for the training of SeRanet.

The website provides high quality photos under Creative Commons Zero (CC0) license.

Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,

After extract, copy this to data/training_images to start your own training.

Contribution is welcome

The performance of SR for this project is still not matured. You are welcome to suggest any improvement & contribute to this project. If you could get any model which performs better performance, feel free to send me a pull request!