Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images

Introduction

This code is relative to paper.

In the paper, training is done using Caltech and KAIST DB seperately, total 6 stages. However, this implementation trains network using both DB at the same time, total 4 stages, for simplicity, resulting a similar performance.

Also, this code is written based on the MATLAB implementations of ShaoqingRen/faster_rcnn and zhangliliang/RPN_BF.

This code has been tested on Windows7 with MATLAB 2017a.

Requirements

0. Caffe

1. MATLAB

2. GPU: Geforce GTX 1070, etc

Preparation for Training

0. download videos and toolboxes from KAIST Multispectral Pedestrian Detection Benchmark and Caltech Pedestrian Detection Benchmark.

1. extract visible camera images and annotations using each toolbox

2. place KAIST(set00-05, skip=10), Caltech(set00-10, skip=30) training images and annotations in ./datasets/train/

3. place KAIST(set06-11, skip=20) testing images and annotations in ./datasets/test/

4. place the toolbox folders (KAIST, Caltech) in ./external/, and name as toolbox(kaist) and toolbox(caltech), respectively

5. run fetch_data/fetch_caffe_mex_cuda65.m to download a compiled Caffe mex (for Windows only).

6. download ImageNet-pre-trained VGG16(reduced for 7x3 ROI pooling) model(depicted below) from GoogleDrive and place it to ./models/pre_trained_models/vgg_16layers

Training

0. Run startup.m

1. Run faster_rcnn_VGG16.m

Preparation for Testing

0. extract KAIST(set06-11) testing images with skip frame=1 for the fusion of successive images.

1. place these images in ./datasets/skip1/

Testing

0. Run final_test.m to get the result in ./test/faster-rcnn-test3

1. Run plotMR.m to see the graph