Collection of papers, datasets, code and other resources for object detection and tracking using deep learning
- Research Data
- Papers
- Datasets
- Code
- Collections
- Tutorials
- Blogs
Research Data
I use DavidRM Journal for managing my research data for its excellent hierarchical organization, cross-linking and tagging capabilities.
I make available a Journal entry export file that contains tagged and categorized collection of papers, articles and notes about computer vision and deep learning that I have collected over the last few years.
It needs Jounal 8 and can be imported using File -> Import -> Sync from The Journal Export File. My user preferences also need to be imported (File -> Import -> Import User Preferences) before importing the entries for the tagged topics to work correctly. My global options file is also provided for those interested in a dark theme.
Updated: 200711_142021
Papers
Static Detection
Region Proposal
- Scalable Object Detection Using Deep Neural Networks [cvpr14] [pdf] [notes]
- Selective Search for Object Recognition [ijcv2013] [pdf] [notes]
RCNN
- Faster R-CNN Towards Real-Time Object Detection with Region Proposal Networks [tpami17] [pdf] [notes]
- RFCN - Object Detection via Region-based Fully Convolutional Networks [nips16] [Microsoft Research] [pdf] [notes]
- Mask R-CNN [iccv17] [Facebook AI Research] [pdf] [notes] [arxiv] [code (keras)] [code (tensorflow)]
- SNIPER Efficient Multi-Scale Training [ax1812/nips18] [pdf] [notes] [code]
YOLO
- You Only Look Once Unified, Real-Time Object Detection [ax1605] [pdf] [notes]
- YOLO9000 Better, Faster, Stronger [ax1612] [pdf] [notes]
- YOLOv3 An Incremental Improvement [ax1804] [pdf] [notes]
- YOLOv4 Optimal Speed and Accuracy of Object Detection [ax2004] [pdf] [notes] [code]
SSD
- SSD Single Shot MultiBox Detector [ax1612/eccv16] [pdf] [notes]
- DSSD Deconvolutional Single Shot Detector [ax1701] [pdf] [notes]
RetinaNet
- Feature Pyramid Networks for Object Detection [ax1704] [pdf] [notes]
- Focal Loss for Dense Object Detection [ax180207/iccv17] [pdf] [notes]
Anchor Free
- FoveaBox: Beyond Anchor-based Object Detector [ax1904] [pdf] [notes] [code]
- CornerNet: Detecting Objects as Paired Keypoints [ax1903/ijcv19] [pdf] [notes] [code]
- FCOS Fully Convolutional One-Stage Object Detection [ax1908/iccv19] [pdf] [notes] [code] [code/FCOS_PLUS] [code/VoVNet] [code/HRNet] [code/NAS]
- Feature Selective Anchor-Free Module for Single-Shot Object Detection [ax1903/cvpr19] [pdf] [notes] [code]
- Bottom-up object detection by grouping extreme and center points [ax1901] [pdf] [notes] [code]
- Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection [ax1912/cvpr20] [pdf] [notes] [code]
- End-to-end object detection with Transformers [ax200528] [pdf] [notes] [code]
- Objects as Points [ax1904] [pdf] [notes] [code]
- RepPoints Point Set Representation for Object Detection [iccv19] [pdf] [notes] [code]
Misc
- OverFeat Integrated Recognition, Localization and Detection using Convolutional Networks [ax1402/iclr14] [pdf] [notes]
- LSDA Large scale detection through adaptation [ax1411/nips14] [pdf] [notes]
- Acquisition of Localization Confidence for Accurate Object Detection [ax1807/eccv18] [pdf] [notes] [code]
- EfficientDet: Scalable and Efficient Object Detection [cvpr20] [pdf]
- Generalized Intersection over Union A Metric and A Loss for Bounding Box Regression [ax1902/cvpr19] [pdf] [notes] [code] [project]
Video Detection
Tubelet
- Object Detection from Video Tubelets with Convolutional Neural Networks [cvpr16] [pdf] [notes]
- Object Detection in Videos with Tubelet Proposal Networks [ax1704/cvpr17] [pdf] [notes]
FGFA
- Deep Feature Flow for Video Recognition [cvpr17] [Microsoft Research] [pdf] [arxiv] [code]
- Flow-Guided Feature Aggregation for Video Object Detection [ax1708/iccv17] [pdf] [notes]
- Towards High Performance Video Object Detection [ax1711] [Microsoft] [pdf] [notes]
RNN
- Online Video Object Detection using Association LSTM [iccv17] [pdf] [notes]
- Context Matters Refining Object Detection in Video with Recurrent Neural Networks [bmvc16] [pdf] [notes]
Multi Object Tracking
Joint-Detection
Identity Embedding
-
MOTS Multi-Object Tracking and Segmentation [cvpr19] [pdf] [notes] [code] [project/data]
-
Towards Real-Time Multi-Object Tracking [ax1909] [pdf] [notes]
-
A Simple Baseline for Multi-Object Tracking [ax2004] [pdf] [notes] [code]
-
Integrated Object Detection and Tracking with Tracklet-Conditioned Detection [ax1811] [pdf] [notes]
Association
Deep Learning
-
Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism [ax1708/iccv17] [pdf] [arxiv] [notes]
-
Online multi-object tracking with dual matching attention networks [ax1902/eccv18] [pdf] [arxiv] [notes] [code]
-
FAMNet Joint Learning of Feature, Affinity and Multi-Dimensional Assignment for Online Multiple Object Tracking [iccv19] [pdf] [notes]
-
Exploit the Connectivity: Multi-Object Tracking with TrackletNet [ax1811/mm19] [pdf] [notes]
-
Tracking without bells and whistles [ax1903/iccv19] [pdf] [notes] [code] [pytorch]
RNN
- Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies [ax1704/iccv17] [Stanford] [pdf] [notes] [arxiv] [project],
- Multi-object Tracking with Neural Gating Using Bilinear LSTM [eccv18] [pdf] [notes]
- Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking [cvpr19] [pdf] [notes] [code]
Unsupervised Learning
- Unsupervised Person Re-identification by Deep Learning Tracklet Association [ax1809/eccv18] [pdf] [notes]
- Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers [ax1809/cvpr19] [pdf] [arxiv] [notes] [code]
- Simple Unsupervised Multi-Object Tracking [ax2006] [pdf] [notes]
Reinforcement Learning
- Learning to Track: Online Multi-object Tracking by Decision Making [iccv15] [Stanford] [pdf] [notes] [code (matlab)] [project]
- Collaborative Deep Reinforcement Learning for Multi-Object Tracking [eccv18] [pdf] [notes]
Network Flow
- Near-Online Multi-target Tracking with Aggregated Local Flow Descriptor [iccv15] [NEC Labs] [pdf] [author] [notes]
- Deep Network Flow for Multi-Object Tracking [cvpr17] [NEC Labs] [pdf] [supplementary] [notes]
- Learning a Neural Solver for Multiple Object Tracking [ax1912/cvpr20] [pdf] [notes] [code]
Graph Optimization
- A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects [ax1607] [highest MT on MOT2015] [University of Freiburg, Germany] [pdf] [arxiv] [author] [notes]
Baseline
- Simple Online and Realtime Tracking [icip16] [pdf] [notes] [code]
- High-Speed Tracking-by-Detection Without Using Image Information [avss17] [pdf] [notes] [code]
Metrics
Single Object Tracking
Reinforcement Learning
- Deep Reinforcement Learning for Visual Object Tracking in Videos [ax1704] [USC-Santa Barbara, Samsung Research] [pdf] [arxiv] [author] [notes]
- Visual Tracking by Reinforced Decision Making [ax1702] [Seoul National University, Chung-Ang University] [pdf] [arxiv] [author] [notes]
- Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning [cvpr17] [Seoul National University] [pdf] [supplementary] [project] [notes] [code]
- End-to-end Active Object Tracking via Reinforcement Learning [ax1705] [Peking University, Tencent AI Lab] [pdf] [arxiv]
Siamese
- Fully-Convolutional Siamese Networks for Object Tracking [eccv16] [pdf] [project] [notes]
- High Performance Visual Tracking with Siamese Region Proposal Network [cvpr18] [pdf] [author] [notes]
- Siam R-CNN Visual Tracking by Re-Detection [cvpr20] [pdf] [notes] [project] [code]
Correlation
- ATOM Accurate Tracking by Overlap Maximization [cvpr19] [pdf] [notes] [code]
- DiMP Learning Discriminative Model Prediction for Tracking [iccv19] [pdf] [notes] [code]
- D3S – A Discriminative Single Shot Segmentation Tracker [cvpr20] [pdf] [notes] [code]
Misc
Deep Learning
Synthetic Gradients
- Decoupled Neural Interfaces using Synthetic Gradients [ax1608] [pdf] [notes]
- Understanding Synthetic Gradients and Decoupled Neural Interfaces [ax1703] [pdf] [notes]
Efficient
Unsupervised Learning
Interpolation
- Video Frame Interpolation via Adaptive Convolution [cvpr17 / iccv17] [pdf (cvpr17)] [pdf (iccv17)] [ppt]
Autoencoder
Variational
- beta-VAE Learning Basic Visual Concepts with a Constrained Variational Framework [iclr17] [pdf] [notes]
- Disentangling by Factorising [ax1806] [pdf] [notes]
Datasets
Multi Object Tracking
- IDOT
- UA-DETRAC Benchmark Suite
- GRAM Road-Traffic Monitoring
- Ko-PER Intersection Dataset
- TRANCOS
- Urban Tracker
- DARPA VIVID / PETS 2005 [Non stationary camera]
- KIT-AKS [No ground truth]
- CBCL StreetScenes Challenge Framework [No top down viewpoint]
- MOT 2015 [mostly street level viewpoint]
- MOT 2016 [mostly street level viewpoint]
- MOT 2017 [mostly street level viewpoint]
- MOT 2020 [mostly top down viewpoint]
- MOTS: Multi-Object Tracking and Segmentation [MOT and KITTI]
- CVPR 2019 [mostly street level viewpoint]
- PETS 2009 [No vehicles]
- PETS 2017 [Low density] [mostly pedestrians]
- DukeMTMC [multi camera] [static background] [pedestrians] [above-street level viewpoint] [website not working]
- KITTI Tracking Dataset [No top down viewpoint] [non stationary camera]
- The WILDTRACK Seven-Camera HD Dataset [pedestrian detection and tracking]
- 3D Traffic Scene Understanding from Movable Platforms [intersection traffic] [stereo setup] [moving camera]
- LOST : Longterm Observation of Scenes with Tracks [top down and street level viewpoint] [no ground truth]
- JTA [top down and street level viewpoint] [synthetic/GTA 5] [pedestrian] [3D annotations]
- PathTrack: Fast Trajectory Annotation with Path Supervision [top down and street level viewpoint] [iccv17] [pedestrian]
- CityFlow [pole mounted] [intersections] [vehicles] [re-id] [cvpr19]
- JackRabbot Dataset [RGBD] [head-on][indoor/outdoor][stanford]
- TAO: A Large-Scale Benchmark for Tracking Any Object [eccv20] [code]
- Edinburgh office monitoring video dataset [indoors][long term][mostly static people]
- Waymo Open Dataset [outdoors][vehicles]
UAV
- Stanford Drone Dataset
- UAVDT - The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking [uav] [intersections/highways] [vehicles] [eccv18]
- VisDrone
Synthetic
- MNIST-MOT / MNIST-Sprites [script generated] [cvpr19]
- TUB Multi-Object and Multi-Camera Tracking Dataset [avss16]
- Virtual KITTI [arxiv] [cvpr16] [link seems broken]
Microscopy / Cell Tracking
- Cell Tracking Challenge [nature methods/2017]
- CTMC: Cell Tracking with Mitosis Detection Dataset Challenge [cvprw20] [MOT]
Single Object Tracking
- TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild [eccv18]
- LaSOT: Large-scale Single Object Tracking [cvpr19]
- Need for speed: A benchmark for higher frame rate object tracking [iccv17]
- Long-term Tracking in the Wild A Benchmark [eccv18]
- UAV123: A benchmark and simulator for UAV tracking [eccv16] [project]
- Sim4CV A Photo-Realistic Simulator for Computer Vision Applications [ijcv18]
- CDTB: A Color and Depth Visual Object Tracking and Benchmark [iccv19] [RGBD]
- Temple Color 128 - Color Tracking Benchmark [tip15]
Video Detection
Video Understanding / Activity Recognition
- YouTube-8M
- AVA: A Video Dataset of Atomic Visual Action
- VIRAT Video Dataset
- Kinetics Action Recognition Dataset
Static Detection
- PASCAL Visual Object Classes
- A Large-Scale Dataset for Vehicle Re-Identification in the Wild [cvpr19]
- Object Detection-based annotations for some frames of the VIRAT dataset
- MIO-TCD: A new benchmark dataset for vehicle classification and localization [tip18]
- Tiny ImageNet
Animals
- Wildlife Image and Localization Dataset (species and bounding box labels) [wacv18]
- Stanford Dogs Dataset [cvpr11]
- Oxford-IIIT Pet Dataset [cvpr12]
- Caltech-UCSD Birds 200 [rough segmentation] [attributes]
- Gold Standard Snapshot Serengeti Bounding Box Coordinates
Boundary Detection
Static Segmentation
- COCO - Common Objects in Context
- Open Images
- ADE20K [cvpr17]
- SYNTHIA [cvpr16]
- UC Berkeley Computer Vision Group - Contour Detection and Image Segmentation
Video Segmentation
- DAVIS: Densely Annotated VIdeo Segmentation
- Mapillary Vistas Dataset [street scenes] [semi-free]
- BDD100K [street scenes] [autonomous driving]
- ApolloScape [street scenes] [autonomous driving]
- Cityscapes [street scenes] [instance-level]
- YouTube-VOS [iccv19]
Classification
- ImageNet Large Scale Visual Recognition Competition 2012
- Animals with Attributes 2
- CompCars Dataset
- ObjectNet [only test set]
Optical Flow
Motion Prediction
Code
General Vision
- Gluon CV Toolkit [mxnet] [pytorch]
- OpenMMLab Computer Vision Foundation [pytorch]
Multi Object Tracking
Frameworks
General
- Globally-optimal greedy algorithms for tracking a variable number of objects [cvpr11] [matlab] [author]
- Continuous Energy Minimization for Multitarget Tracking [cvpr11 / iccv11 / tpami 2014] [matlab]
- Discrete-Continuous Energy Minimization for Multi-Target Tracking [cvpr12] [matlab] [project]
- The way they move: Tracking multiple targets with similar appearance [iccv13] [matlab]
- 3D Traffic Scene Understanding from Movable Platforms [2d_tracking] [pami14/kit13/iccv13/nips11] [c++/matlab]
- Multiple target tracking based on undirected hierarchical relation hypergraph [cvpr14] [C++] [author]
- Robust online multi-object tracking based on tracklet confidence and online discriminative appearance learning [cvpr14] [matlab] (project)
- Learning to Track: Online Multi-Object Tracking by Decision Making [iccv15] [matlab]
- Joint Tracking and Segmentation of Multiple Targets [cvpr15] [matlab]
- Multiple Hypothesis Tracking Revisited [iccv15] [highest MT on MOT2015 among open source trackers] [matlab]
- Combined Image- and World-Space Tracking in Traffic Scenes [icra 2017] [c++]
- Online Multi-Target Tracking with Recurrent Neural Networks [aaai17] [lua/torch7]
- Real-Time Multiple Object Tracking - A Study on the Importance of Speed [ax1710/masters thesis] [c++]
- Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking [icra18] [matlab]
- Online Multi-Object Tracking with Dual Matching Attention Network [eccv18] [matlab/tensorflow]
- TrackR-CNN - Multi-Object Tracking and Segmentation [cvpr19] [tensorflow] [project]
- Eliminating Exposure Bias and Metric Mismatch in Multiple Object Tracking [cvpr19] [tensorflow]
- Robust Multi-Modality Multi-Object Tracking [iccv19] [pytorch]
- Towards Real-Time Multi-Object Tracking / Joint Detection and Embedding [ax1909] [pytorch] [CMU]
- Deep Affinity Network for Multiple Object Tracking [tpami19] [pytorch]
- Tracking without bells and whistles [iccv19] [pytorch]
- Lifted Disjoint Paths with Application in Multiple Object Tracking [icml20] [matlab] [mot15#1,mot16 #3,mot17#2]
- Learning a Neural Solver for Multiple Object Tracking [cvpr20] [pytorch] [mot15#2]
- Tracking Objects as Points [ax2004] [pytorch]
- Quasi-Dense Similarity Learning for Multiple Object Tracking [ax2006] [pytorch]
- DEFT: Detection Embeddings for Tracking [ax2102] [pytorch]
- How To Train Your Deep Multi-Object Tracker [ax1906/cvpr20] [pytorch] [traktor/gitlab]
Baseline
- Simple Online and Realtime Tracking [icip 2016] [python]
- Deep SORT : Simple Online Realtime Tracking with a Deep Association Metric [icip17] [python]
- High-Speed Tracking-by-Detection Without Using Image Information [avss17] [python]
- A simple baseline for one-shot multi-object tracking [ax2004] [pytorch] [winner of mot15,16,17,20]
Siamese
- SiamMOT: Siamese Multi-Object Tracking [ax2105] [pytorch]
Unsupervised
- Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers [cvpr19] [python/c++/pytorch]
Re-ID
- Torchreid: Deep learning person re-identification in PyTorch [ax1910] [pytorch]
- SMOT: Single-Shot Multi Object Tracking [ax2010] [pytorch] [gluon-cv]
- FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking [ax2004] [pytorch] [microsoft] [BDD100K] [face tracking]
- Rethinking the competition between detection and ReID in Multi-Object Tracking [ax2010] [pytorch]
Frameworks
Graph NN
- Joint Object Detection and Multi-Object Tracking with Graph Neural Networks [ax2006/ icra21] [pytorch]
Microscopy / cell tracking
- Baxter Algorithms / Viterbi Tracking [tmi14] [matlab]
- Deepcell: Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning [biorxiv1910] [tensorflow]
3D
- 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics [iros20/eccvw20] [pytorch]
- GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning [iros20/eccvw20] [pytorch]
Metrics
- HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking [cvpr20] [python]
Single Object Tracking
- A collection of common tracking algorithms (2003-2012) [c++/matlab]
- SenseTime Research platform for single object tracking, implementing algorithms like SiamRPN and SiamMask [pytorch]
- In Defense of Color-based Model-free Tracking [cvpr15] [c++]
- Hierarchical Convolutional Features for Visual Tracking [iccv15] [matlab]
- Visual Tracking with Fully Convolutional Networks [iccv15] [matlab]
- Hierarchical Convolutional Features for Visual Tracking [iccv15] [matlab]
- DeepTracking: Seeing Beyond Seeing Using Recurrent Neural Networks [aaai16] [torch 7]
- Learning Multi-Domain Convolutional Neural Networks for Visual Tracking [cvpr16] [vot2015 winner] [matlab/matconvnet] [pytorch]
- Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking [eccv 2016] [matlab]
- Fully-Convolutional Siamese Networks for Object Tracking [eccvw 2016] [matlab/matconvnet] [project] [pytorch] [pytorch (only training)]
- DCFNet: Discriminant Correlation Filters Network for Visual Tracking [ax1704] [matlab/matconvnet] [pytorch]
- End-to-end representation learning for Correlation Filter based tracking [cvpr17] [matlab/matconvnet] [tensorflow/inference_only] [project]
- Dual Deep Network for Visual Tracking [tip1704] [caffe]
- A simplified PyTorch implementation of Siamese networks for tracking: SiamFC, SiamRPN, SiamRPN++, SiamVGG, SiamDW, SiamRPN-VGG [pytorch]
- RATM: Recurrent Attentive Tracking Model [cvprw17] [python]
- ROLO : Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking [iscas 2017] [tensorfow]
- ECO: Efficient Convolution Operators for Tracking [cvpr17] [matlab] [python/cuda] [pytorch]
- Action-Decision Networks for Visual Tracking with Deep Reinforcement Learning [cvpr17] [tensorflow]
- Detect to Track and Track to Detect [iccv17] [matlab]
- Meta-Tracker: Fast and Robust Online Adaptation for Visual Object Trackers [eccv18] [pytorch]
- Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking [cvpr18] [matlab]
- High Performance Visual Tracking with Siamese Region Proposal Network [cvpr18] [pytorch/195] [pytorch/313] [pytorch/no_train/104] [pytorch/177]
- Distractor-aware Siamese Networks for Visual Object Tracking [eccv18] [vot18 winner] [pytorch]
- VITAL: VIsual Tracking via Adversarial Learning [cvpr18] [matlab] [pytorch] [project]
- Fast Online Object Tracking and Segmentation: A Unifying Approach (SiamMask) [cvpr19] [pytorch] [project]
- PyTracking: A general python framework for training and running visual object trackers, based on PyTorch [ECO/ATOM/DiMP/PrDiMP] [cvpr17/cvpr19/iccv19/cvpr20] [pytorch]
- Unsupervised Deep Tracking [cvpr19] [matlab/matconvnet] [pytorch]
- Deeper and Wider Siamese Networks for Real-Time Visual Tracking [cvpr19] [pytorch]
- GradNet: Gradient-Guided Network for Visual Object Tracking [iccv19] [tensorflow]
- `Skimming-Perusal' Tracking: A Framework for Real-Time and Robust Long-term Tracking [iccv19] [tensorflow]
- Learning Aberrance Repressed Correlation Filters for Real-Time UAV Tracking [iccv19] [matlab]
- Learning the Model Update for Siamese Trackers [iccv19] [pytorch]
- SPM-Tracker: Series-Parallel Matching for Real-Time Visual Object Tracking [cvpr19] [pytorch] [inference-only]
- Joint Group Feature Selection and Discriminative Filter Learning for Robust Visual Object Tracking [iccv19] [matlab]
- Siam R-CNN: Visual Tracking by Re-Detection [cvpr20] [tensorflow]
- D3S - Discriminative Single Shot Segmentation Tracker [cvpr20] [pytorch/pytracking]
- Discriminative and Robust Online Learning for Siamese Visual Tracking [aaai20] [pytorch/pysot]
- Siamese Box Adaptive Network for Visual Tracking [cvpr20] [pytorch/pysot]
- Ocean: Object-aware Anchor-free Tracking [ax2010] [pytorch]
GUI Application / Large Scale Tracking / Animals
- BioTracker An Open-Source Computer Vision Framework for Visual Animal Tracking[opencv/c++]
- Tracktor: Image‐based automated tracking of animal movement and behaviour[opencv/c++]
- MARGO (Massively Automated Real-time GUI for Object-tracking), a platform for high-throughput ethology[matlab]
- idtracker.ai: Tracking all individuals in large collectives of unmarked animals [tensorflow] [project]
Video Detection
- Flow-Guided Feature Aggregation for Video Object Detection [nips16 / iccv17] [mxnet]
- T-CNN: Tubelets with Convolution Neural Networks [cvpr16] [python]
- TPN: Tubelet Proposal Network [cvpr17] [python]
- Deep Feature Flow for Video Recognition [cvpr17] [mxnet]
- Mobile Video Object Detection with Temporally-Aware Feature Maps [cvpr18] [Google] [tensorflow]
Action Detection
Frameworks
Static Detection and Matching
Frameworks
- Tensorflow object detection API [tensorflow]
- Detectron2 [pytorch]
- Detectron [pytorch]
- Open MMLab Detection Toolbox with PyTorch [pytorch]
Region Proposal
- MCG : Multiscale Combinatorial Grouping - Object Proposals and Segmentation (project) [tpami16/cvpr14] [python]
- COB : Convolutional Oriented Boundaries (project) [tpami18/eccv16] [matlab/caffe]
FPN
- Feature Pyramid Networks for Object Detection [caffe/python]
RCNN
- RFCN (author) [caffe/matlab]
- RFCN-tensorflow [tensorflow]
- PVANet: Lightweight Deep Neural Networks for Real-time Object Detection [intel] [emdnn16(nips16)]
- Mask R-CNN [tensorflow] [keras]
- Light-head R-CNN [cvpr18] [tensorflow]
- Evolving Boxes for Fast Vehicle Detection [icme18] [caffe/python]
- Cascade R-CNN (cvpr18) [detectron] [caffe]
- A MultiPath Network for Object Detection [torch] [bmvc16] [facebook]
- SNIPER: Efficient Multi-Scale Training/An Analysis of Scale Invariance in Object Detection-SNIP [nips18/cvpr18] [mxnet]
SSD
- SSD-Tensorflow [tensorflow]
- SSD-Tensorflow (tf.estimator) [tensorflow]
- SSD-Tensorflow (tf.slim) [tensorflow]
- SSD-Keras [keras]
- SSD-Pytorch [pytorch]
- Enhanced SSD with Feature Fusion and Visual Reasoning [nca18] [tensorflow]
- RefineDet - Single-Shot Refinement Neural Network for Object Detection [cvpr18] [caffe]
RetinaNet
- 9.277.41 [pytorch]
- 31.857.212 [pytorch]
- 25.274.84 [pytorch] [nvidia]
- 22.869.302 [pytorch]
YOLO
- Darknet: Convolutional Neural Networks [c/python]
- YOLO9000: Better, Faster, Stronger - Real-Time Object Detection. 9000 classes! [c/python]
- Darkflow [tensorflow]
- Pytorch Yolov2 [pytorch]
- Yolo-v3 and Yolo-v2 for Windows and Linux [c/python]
- YOLOv3 in PyTorch [pytorch]
- pytorch-yolo-v3 [pytorch] [no training] [tutorial]
- YOLOv3_TensorFlow [tensorflow]
- tensorflow-yolo-v3 [tensorflow slim]
- tensorflow-yolov3 [tensorflow slim]
- keras-yolov3 [keras]
- YOLOv4 [darknet - c/python] [tensorflow] [pytorch/711] [pytorch/ONNX/TensorRT/1.9k] [pytorch 3D]
- YOLOv5 [pytorch]
Anchor Free
- FoveaBox: Beyond Anchor-based Object Detector [ax1904] [pytorch/mmdetection]
- Cornernet: Detecting objects as paired keypoints [ax1903/eccv18] [pytorch]
- FCOS: Fully Convolutional One-Stage Object Detection [iccv19] [pytorch] [VoVNet] [HRNet] [NAS] [FCOS_PLUS]
- Feature Selective Anchor-Free Module for Single-Shot Object Detection [cvpr19] [pytorch]
- CenterNet: Objects as Points [ax1904] [pytorch]
- Bottom-up Object Detection by Grouping Extreme and Center Points, [cvpr19] [pytorch]
- RepPoints Point Set Representation for Object Detection [iccv19] [pytorch] [microsoft]
- DE⫶TR: End-to-End Object Detection with Transformers [ax200528] [pytorch] [facebook]
- Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection [cvpr20] [pytorch]
Misc
- Relation Networks for Object Detection [cvpr18] [mxnet]
- DeNet: Scalable Real-time Object Detection with Directed Sparse Sampling [iccv17(poster)] [theano]
- Multi-scale Location-aware Kernel Representation for Object Detection [cvpr18] [caffe/python]
Matching
Boundary Detection
- Holistically-Nested Edge Detection (HED) (iccv15) [caffe]
- Edge-Detection-using-Deep-Learning (HED) [tensorflow]
- Holistically-Nested Edge Detection (HED) in OpenCV [python/c++]
- Crisp Boundary Detection Using Pointwise Mutual Information (eccv14) [matlab]
- Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection [wacv20] tensorflow pytorch
Text Detection
- Real-time Scene Text Detection with Differentiable Binarization [pytorch] [aaai20]
Frameworks
3D Detection
Frameworks
- OpenMMLab's next-generation platform for general 3D object detection [pytorch]
- OpenPCDet Toolbox for LiDAR-based 3D Object Detection [pytorch]
Optical Flow
- FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks (cvpr17) [caffe] [pytorch/nvidia]
- SPyNet: Spatial Pyramid Network for Optical Flow (cvpr17) [lua] [pytorch]
- Guided Optical Flow Learning (cvprw17) [caffe] [tensorflow]
- Fast Optical Flow using Dense Inverse Search (DIS) [eccv16] [C++]
- A Filter Formulation for Computing Real Time Optical Flow [ral16] [c++/cuda - matlab,python wrappers]
- PatchBatch - a Batch Augmented Loss for Optical Flow [cvpr16] [python/theano]
- Piecewise Rigid Scene Flow [iccv13/eccv14/ijcv15] [c++/matlab]
- DeepFlow v2 [iccv13] [c++/python/matlab], [project]
- An Evaluation of Data Costs for Optical Flow [gcpr13] [matlab]
Frameworks
Instance Segmentation
- Fully Convolutional Instance-aware Semantic Segmentation [cvpr17] [coco16 winner] [mxnet]
- Instance-aware Semantic Segmentation via Multi-task Network Cascades [cvpr16] [caffe] [coco15 winner]
- DeepMask/SharpMask [nips15/eccv16] [facebook] [torch] [tensorflow] [pytorch/deepmask]
- Simultaneous Detection and Segmentation [eccv14] [matlab] [project]
- PANet [cvpr18] [pytorch]
- RetinaMask [arxviv1901] [pytorch]
- Mask Scoring R-CNN [cvpr19] [pytorch]
- DeepMAC [ax2104] [tensorflow]
- Swin Transformer [iccv21] [pytorch] [microsoft]
Frameworks
- Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch [pytorch] [facebook]
- PaddleDetection, Object detection and instance segmentation toolkit based on PaddlePaddle. [2019]
Semantic Segmentation
- Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation [cvpr18] [spotlight] [pytorch]
- Few-shot Segmentation Propagation with Guided Networks [ax1806] [pytorch] [incomplete]
- Pytorch-segmentation-toolbox [DeeplabV3 and PSPNet] [pytorch]
- DeepLab [tensorflow]
- Auto-DeepLab [pytorch]
- DeepLab v3+ [pytorch]
- Deep Extreme Cut (DEXTR): From Extreme Points to Object Segmentation[cvpr18][project] [pytorch]
- FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation[ax1903][project] [pytorch]
Frameworks
Polyp
- PraNet: Parallel Reverse Attention Network for Polyp Segmentation[miccai20]
- PHarDNet-MSEG: A Simple Encoder-Decoder Polyp Segmentation Neural Network that Achieves over 0.9 Mean Dice and 86 FPS[ax2101]
Panoptic Segmentation
- Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation [cvpr20] [pytorch]
Video Segmentation
- Improving Semantic Segmentation via Video Prediction and Label Relaxation [cvpr19] [pytorch] [nvidia]
- PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation [accv18/cvprw18/eccvw18] [tensorflow]
- MaskTrackRCNN for video instance segmentation [iccv19] [pytorch/detectron]
- Video Swin Transformer [iccv19] [pytorch/detectron]
Panoptic Video Segmentation
- ViP-DeepLab [cvpr21]
Motion Prediction
- Self-Supervised Learning via Conditional Motion Propagation [cvpr19] [pytorch]
- A Neural Temporal Model for Human Motion Prediction [cvpr19] [tensorflow]
- Learning Trajectory Dependencies for Human Motion Prediction [iccv19] [pytorch]
- Structural-RNN: Deep Learning on Spatio-Temporal Graphs [cvpr15] [tensorflow]
- A Keras multi-input multi-output LSTM-based RNN for object trajectory forecasting [keras]
- Transformer Networks for Trajectory Forecasting [ax2003] [pytorch]
- Regularizing neural networks for future trajectory prediction via IRL framework [ietcv1907] [tensorflow]
- Peeking into the Future: Predicting Future Person Activities and Locations in Videos [cvpr19] [tensorflow]
- DAG-Net: Double Attentive Graph Neural Network for Trajectory Forecasting [ax200526] [pytorch]
- MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic [ax200405] [tensorflow]
- Human Trajectory Prediction in Socially Interacting Crowds Using a CNN-based Architecture [pytorch]
- A tool set for trajectory prediction, ready for pip install [icai19/wacv19] [pytorch]
- RobustTP: End-to-End Trajectory Prediction for Heterogeneous Road-Agents in Dense Traffic with Noisy Sensor Inputs [acmcscs19] [pytorch/tensorflow]
- The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction [cvpr20] [dummy]
- Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction [cvpr19] [tensorflow]
- Adversarial Loss for Human Trajectory Prediction [hEART19] [pytorch]
- Social GAN: SSocially Acceptable Trajectories with Generative Adversarial Networks [cvpr18] [pytorch]
- Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs [ax1912] [pytorch]
- Study of attention mechanisms for trajectory prediction in Deep Learning [msc thesis] [python]
- A python implementation of multi-model estimation algorithm for trajectory tracking and prediction, research project from BMW ABSOLUT self-driving bus project. [python]
- Prediciting Human Trajectories [theano]
- Implementation of Recurrent Neural Networks for future trajectory prediction of pedestrians [pytorch]
Pose Estimation
Frameworks
Autoencoders
- β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework [iclr17] [deepmind] [tensorflow] [tensorflow] [pytorch]
- Disentangling by Factorising [ax1806] [pytorch]
Classification
- Learning Efficient Convolutional Networks Through Network Slimming [iccv17] [pytorch]
Frameworks
Deep RL
Annotation
- LabelImg
- ByLabel: A Boundary Based Semi-Automatic Image Annotation Tool
- Bounding Box Editor and Exporter
- VGG Image Annotator
- Visual Object Tagging Tool: An electron app for building end to end Object Detection Models from Images and Videos
- PixelAnnotationTool
- labelme : Image Polygonal Annotation with Python (polygon, rectangle, circle, line, point and image-level flag annotation)
- VATIC - Video Annotation Tool from Irvine, California) [ijcv12] [project]
- Computer Vision Annotation Tool (CVAT)
- Image labelling tool
- Labelbox [paid]
- RectLabel An image annotation tool to label images for bounding box object detection and segmentation. [paid]
- Onepanel: Production scale vision AI platform with fully integrated components for model building, automated labeling, data processing and model training pipelines. [docs]
Editing
Augmentation
- Augmentor: Image augmentation library in Python for machine learning
- Albumentations: Fast image augmentation library and easy to use wrapper around other libraries
- imgaug: Image augmentation for machine learning experiments
- solt: Image Streaming over lightweight data transformations
Deep Learning
Class Imbalance
- Imbalanced Dataset Sampler [pytorch]
- Iterable dataset resampling in PyTorch [pytorch]
Few shot learning
Unsupervised learning
Collections
Datasets
- Awesome Public Datasets
- List of traffic surveillance datasets
- Machine learning datasets: A list of the biggest machine learning datasets from across the web
- Labeled Information Library of Alexandria: Biology and Conservation [other conservation data sets]
- THOTH: Data Sets & Images
- Google AI Datasets
- Google Cloud Storage public datasets
- Microsoft Research Open Data
- Earth Engine Data Catalog
- Registry of Open Data on AWS
- Kaggle Datasets
- CVonline: Image Databases
- Synthetic for Computer Vision: A list of synthetic dataset and tools for computer vision
- pgram machine learning datasets
- pgram vision datasets
Deep Learning
Static Detection
Video Detection
Single Object Tracking
- Visual Tracking Paper List
- List of deep learning based tracking papers
- List of single object trackers with results on OTB
- Collection of Correlation Filter based trackers with links to papers, codes, etc
- VOT2018 Trackers repository
- CUHK Datasets
- A Summary of CVPR19 Visual Tracking Papers
- Visual Trackers for Single Object
Multi Object Tracking
- List of multi object tracking papers
- A collection of Multiple Object Tracking (MOT) papers in recent years, with notes
- Papers with Code : Multiple Object Tracking
- Paper list and source code for multi-object-tracking
Segmentation
- Segmentation Papers and Code
- Segmentation.X : Papers and Benchmarks about semantic segmentation, instance segmentation, panoptic segmentation and video segmentation
- Instance Segmentation Papers with Code
Motion Prediction
- Awesome-Trajectory-Prediction
- Awesome Interaction-aware Behavior and Trajectory Prediction
- Human Trajectory Prediction Datasets
Deep Compressed Sensing
Misc
- Papers With Code : the latest in machine learning
- Awesome Deep Ecology
- List of Matlab frameworks, libraries and software
- Face Recognition
- A Month of Machine Learning Paper Summaries
- Awesome-model-compression-and-acceleration
- Model-Compression-Papers
Tutorials
Multi Object Tracking
Static Detection
- End-to-end object detection with Transformers
- Deep Learning for Object Detection: A Comprehensive Review
- Review of Deep Learning Algorithms for Object Detection
- A Simple Guide to the Versions of the Inception Network
- R-CNN, Fast R-CNN, Faster R-CNN, YOLO - Object Detection Algorithms
- A gentle guide to deep learning object detection
- The intuition behind RetinaNet
- YOLO—You only look once, real time object detection explained
- Understanding Feature Pyramid Networks for object detection (FPN)
- Fast object detection with SqueezeDet on Keras
- Region of interest pooling explained
Video Detection
Instance Segmentation
- Splash of Color: Instance Segmentation with Mask R-CNN and TensorFlow
- Simple Understanding of Mask RCNN
- Learning to Segment
- Analyzing The Papers Behind Facebook's Computer Vision Approach
- Review: MNC — Multi-task Network Cascade, Winner in 2015 COCO Segmentation
- Review: FCIS — Winner in 2016 COCO Segmentation
- Review: InstanceFCN — Instance-Sensitive Score Maps
Deep Learning
Optimization
Class Imbalance
- Learning from imbalanced data
- Learning from Imbalanced Classes
- Handling imbalanced datasets in machine learning [medium]
- How to handle Class Imbalance Problem [medium]
- Dealing with Imbalanced Data [towardsdatascience]
- How to Handle Imbalanced Classes in Machine Learning [elitedatascience]
- 7 Techniques to Handle Imbalanced Data [kdnuggets]
- 10 Techniques to deal with Imbalanced Classes in Machine Learning [analyticsvidhya]
RNN
Deep RL
Autoencoders
- Guide to Autoencoders
- Applied Deep Learning - Part 3: Autoencoders
- Denoising Autoencoders
- Stacked Denoising Autoencoders
- A Gentle Introduction to LSTM Autoencoders
- Variational Autoencoder in TensorFlow
- Variational Autoencoders with Tensorflow Probability Layers
Blogs
- Facebook AI
- Google AI
- Google DeepMind
- Deep Learning Wizard
- Towards Data Science
- Jay Alammar : Visualizing machine learning one concept at a time
- Inside Machine Learning: Deep-dive articles about machine learning, cloud, and data. Curated by IBM
- colah's blog
- Jeremy Jordan
- Silicon Valley Data Science
- Illarion’s Notes