MSG-Net (Multi-scale guidance network)
Multi-scale guidance network
This repository (https://github.com/twhui/MSG-Net) is the offical release of MSG-Net for our paper Depth Map Super-Resolution by Deep Multi-Scale Guidance in ECCV16. It comes with four trained networks (x2, x4, x8, and x16), one hole-filled RGBD training set, and three hole-filled RGBD testing sets (A, B, and C).
To the best of our knowledge, MSG-Net is the FIRST convolution neural networkwhich attempts to upsample depth images under multi-scale guidance from the corresponding high-resolution RGB images.
Another repository for MS-Net (without multi-scale guidance) is also available.
For more details, please visit my project page.
License and Citation
This software and associated documentation files (the "Software"), and the research paper (Depth Map Super-Resolution by Deep Multi-Scale Guidance) including but not limited to the figures, and tables (the "Paper") are provided for academic research purposes only and without any warranty. Any commercial use requires my consent. When using any parts of the Software or the Paper in your work, please cite the following paper
@InProceedings{hui16msgnet,
author = {Tak-Wai Hui and Chen Change Loy and and Xiaoou Tang},
title = {Depth Map Super-Resolution by Deep Multi-Scale Guidance},
booktitle = {Proceedings of European Conference on Computer Vision (ECCV)},
pages = {353--369},
year = {2016},
url = {http://mmlab.ie.cuhk.edu.hk/projects/guidance_SR_depth.html}
}Dependency
We train our models using caffe and evaluate the results on Matlab.
Installation and Running
You need to install caffe and remeber to complie matcaffe. You can put the folder MSGNet-release in caffe/examples. Finally, you need to get into the the directory of examples/MSGNet-release/util, and run MSGNet.m.
Training data
Our RGBD training set consists of 58 RGBD images from MPI Sintel depth dataset, and 34 RGBD images from Middlebury dataset. 82 images are used for training and 10 images (frames 1, 20, 28, 58, 64, 66, 69, 73, 75 and 79) are used for validation.
Training sub-images/patches (hdf5 format)
An example for generating training patches of the scale factor 2 can be found here util/gen_train_data_x2.m.
Testing data
Testig set is available at the folder MSGNet-release/testing sets.