Awesome Video-Object-Detection

This is a list of awesome articles about object detection from video.

Datasets

ImageNet VID Challenge

• Site: http://image-net.org/challenges/LSVRC/2017/#vid
• Kagge: https://www.kaggle.com/account/login?returnUrl=%2Fc%2Fimagenet-object-detection-from-video-challenge

VisDrone Challenge

• Site: http://aiskyeye.com/

Paper list

2016

Seq-NMS for Video Object Detection

[Arxiv]

• Date: Feb 2016
• Motivation: Smoothing the final bounding box predictions across time.
• Summary: Constructing a temporal graph from overlapping bounding box detections across the adjacent frames, and using dynamic programming to select bounding box sequences with the highest overall detection score.

T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos

[Arxiv] [Code]

• Date: Apr 2016
• Summary: Using a video object detection pipeline that involves predicting optical flow first, then propagating image level predictions according to the flow, and finally using a tracking algorithm to select temporally consistent high confidence detections.
• Performance: 73.8% mAP on ImageNet VID validation.

Object Detection from Video Tubelets with Convolutional Neural Networks

[Arxiv] [Code]

• Date: Apr 2016

Deep Feature Flow for Video Recognition

[Arxiv] [Code]

• Date: Nov 2016
• Performance: 73.0% mAP on ImageNet VID validation at 29 fps on a Titan X GPU.

2017

Object Detection in Videos with Tubelet Proposal Networks

[Arxiv]

• Date: Feb 2017

Flow-Guided Feature Aggregation for Video Object Detection

[Arxiv] [Code]

• Date: Mar 2017
• Motivation: Producing powerful spatiotemporal features.
• Performance: 76.3% mAP at 1.4 fps or 78.4% (combined with Seq-NMS) at 1.1 fps on ImageNet VID validation on a Titan X GPU.

Detect to Track and Track to Detect

[Arxiv] [Summary] [Code]

• Date: Oct 2017
• Motivation: Smoothing the final bounding box predictions across time.
• Summary: Proposing a ConvNet architecture that solves detection and tracking problems jointly and applying a Viterbi algorithm to link the detections across time.
• Performance: 79.8% mAP on ImageNet VID validation.

Towards High Performance Video Object Detection

[Arxiv]

• Date: Nov 2017
• Motivation: Producing powerful spatiotemporal features.
• Performance: 78.6% mAP on ImageNet VID validation at 13 fps on a Titan X GPU.

Video Object Detection with an Aligned Spatial-Temporal Memory

[Arxiv] [Summary] [Code] [Demo]

• Date: Dec 2017
• Motivation: Producing powerful spatiotemporal features.
• Performance: 80.5% mAP on ImageNet VID validation.

2018

Object Detection in Videos by High Quality Object Linking

[Arxiv]

• Date: Jan 2018

Towards High Performance Video Object Detection for Mobiles

[Arxiv]

• Date: Apr 2018
• Motivation: Producing powerful spatiotemporal features.
• Performance: 60.2% mAP on ImageNet VID validation at 25.6 fps on mobiles.

Optimizing Video Object Detection via a Scale-Time Lattice

[Arxiv] [Summary] [Code]

• Date: Apr 2018
• Performance: 79.4% mAP at 20 fps or 79.0% at 62 fps on ImageNet VID validation on a Titan X GPU.

Object Detection in Video with Spatiotemporal Sampling Networks

[Arxiv] [Summary]

• Date: Mar 2018
• Motivation: Producing powerful spatiotemporal features.
• Performance: 78.9% mAP or 80.4% (combined with Seq-NMS) on ImageNet VID validation.

Fully Motion-Aware Network for Video Object Detection

[Paper] [Summary]

• Date: Stp. 2018
• Motivation: Producing powerful spatiotemporal features.
• Performance: 78.1% mAP or 80.3% (combined with Seq-NMS) on ImageNet VID validation.

Integrated Object Detection and Tracking with Tracklet-Conditioned Detection

[Arxiv] [Summary]

• Date: Nov 2018
• Motivation: Smoothing the final bounding box predictions across time.
• Performance: 83.5% of mAP with FGFA and Deformable ConvNets v2 on ImageNet VID validation.

2019

AdaScale: Towards Real-time Video Object Detection Using Adaptive Scaling

[arXiv]

• Date: Feb 2019
• Motivation: Adaptively rescale the input image resolution to improve both accuracy and speed for video object detection.
• Performance: 75.5% of mAP on ImageNet VID validation for 4 different multi-scale training (600, 480, 360, 240).

Improving Video Object Detection by Seq-Bbox Matching

[pdf]

• Date: Feb 2019
• Motivation: Smoothing the final bounding box predictions across time (box-level method).
• Performance: 80.9% of mAP (offline detection) and 78.2% of mAP (online detection) both at 38 fps on a Titan X GPU.

Comparison table