All Projects → amazon-research → siam-mot

amazon-research / siam-mot

Licence: Apache-2.0 license
SiamMOT: Siamese Multi-Object Tracking

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to siam-mot

Fairmot
[IJCV-2021] FairMOT: On the Fairness of Detection and Re-Identification in Multi-Object Tracking
Stars: ✭ 3,194 (+616.14%)
Mutual labels:  multi-object-tracking
Paddledetection
Object Detection toolkit based on PaddlePaddle. It supports object detection, instance segmentation, multiple object tracking and real-time multi-person keypoint detection.
Stars: ✭ 5,799 (+1200.22%)
Mutual labels:  multi-object-tracking
Multi Object Tracking Paper List
Paper list and source code for multi-object-tracking
Stars: ✭ 1,508 (+238.12%)
Mutual labels:  multi-object-tracking
ByteTrack
ByteTrack: Multi-Object Tracking by Associating Every Detection Box
Stars: ✭ 1,991 (+346.41%)
Mutual labels:  multi-object-tracking
ssd sort
Multi-person tracking with SSD and Sort
Stars: ✭ 86 (-80.72%)
Mutual labels:  multi-object-tracking
multi-camera-pig-tracking
Official Implementation of "Tracking Grow-Finish Pigs Across Large Pens Using Multiple Cameras"
Stars: ✭ 25 (-94.39%)
Mutual labels:  multi-object-tracking
deep sort realtime
A really more real-time adaptation of deep sort
Stars: ✭ 31 (-93.05%)
Mutual labels:  multi-object-tracking
MPLT
Multi-person 3D panoramic localization tracking
Stars: ✭ 27 (-93.95%)
Mutual labels:  multi-object-tracking
awesome-3d-multi-object-tracking-autonomous-driving
A summary and list of open source 3D multi object tracking and datasets at this stage.
Stars: ✭ 16 (-96.41%)
Mutual labels:  multi-object-tracking
UniTrack
[NeurIPS'21] Unified tracking framework with a single appearance model. It supports Single Object Tracking (SOT), Video Object Segmentation (VOS), Multi-Object Tracking (MOT), Multi-Object Tracking and Segmentation (MOTS), Pose Tracking, Video Instance Segmentation (VIS), and class-agnostic MOT (e.g. TAO dataset).
Stars: ✭ 293 (-34.3%)
Mutual labels:  multi-object-tracking
2019-CVPR-AIC-Track-1-UWIPL
Repository for 2019 CVPR AI City Challenge Track 1 from IPL@UW
Stars: ✭ 19 (-95.74%)
Mutual labels:  multi-object-tracking
CrowdFlow
Optical Flow Dataset and Benchmark for Visual Crowd Analysis
Stars: ✭ 87 (-80.49%)
Mutual labels:  multi-object-tracking
yolo deepsort
Fast MOT base on yolo+deepsort, support yolo3 and yolo4
Stars: ✭ 47 (-89.46%)
Mutual labels:  multi-object-tracking
Homography-Based-MOTDT
MOTDT with Homography Matrix for Multi-Object Tracking
Stars: ✭ 21 (-95.29%)
Mutual labels:  multi-object-tracking
PyAnomaly
Useful Toolbox for Anomaly Detection
Stars: ✭ 95 (-78.7%)
Mutual labels:  video-analysis
TSP-PRL
Tree-Structured Policy based Progressive Reinforcement Learning for Temporally Language Grounding in Video (AAAI2020)
Stars: ✭ 39 (-91.26%)
Mutual labels:  video-analysis
CBP
Official Tensorflow Implementation of the AAAI-2020 paper "Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction"
Stars: ✭ 52 (-88.34%)
Mutual labels:  video-analysis

SiamMOT

SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

SiamMOT: Siamese Multi-Object Tracking,
Bing Shuai, Andrew Berneshawi, Xinyu Li, Davide Modolo, Joseph Tighe,

@inproceedings{shuai2021siammot,
  title={SiamMOT: Siamese Multi-Object Tracking},
  author={Shuai, Bing and Berneshawi, Andrew and Li, Xinyu and Modolo, Davide and Tighe, Joseph},
  booktitle={CVPR},
  year={2021}
}

Abstract

In this paper, we focus on improving online multi-object tracking (MOT). In particular, we introduce a region-based Siamese Multi-Object Tracking network, which we name SiamMOT. SiamMOT includes a motion model that estimates the instance’s movement between two frames such that detected instances are associated. To explore how the motion modelling affects its tracking capability, we present two variants of Siamese tracker, one that implicitly models motion and one that models it explicitly. We carry out extensive quantitative experiments on three different MOT datasets: MOT17, TAO-person and Caltech Roadside Pedestrians, showing the importance of motion modelling for MOT and the ability of SiamMOT to substantially outperform the state-of-the-art. Finally, SiamMOT also outperforms the winners of ACM MM’20 HiEve Grand Challenge on HiEve dataset. Moreover, SiamMOT is efficient, and it runs at 17 FPS for 720P videos on a single modern GPU.

Installation

Please refer to INSTALL.md for installation instructions.

Try SiamMOT demo

For demo purposes, we provide two tracking models -- tracking person (visible part) or jointly tracking person and vehicles (bus, car, truck, motorcycle, etc). The person tracking model is trained on COCO-17 and CrowdHuman, while the latter model is trained on COCO-17 and VOC12. Currently, both models used in demos use EMM as its motion model, which performs best among different alternatives.

In order to run the demo, use the following command:

python3 demos/demo.py --demo-video  PATH_TO_DEMO_VIDE --track-class person --dump-video True

You can choose person or person_vehicel for track-class such that person tracking or person/vehicle tracking model is used accordingly.

The model would be automatically downloaded to demos/models, and the visualization of tracking outputs is automatically saved to demos/demo_vis

We also provide several pre-trained models in model_zoos.md that can be used for demo.

Dataset Evaluation and Training

After installation, follow the instructions in DATA.md to setup the datasets. As a sanity check, the models presented in model_zoos.md can be used to for benchmark testing.

Use the following command to train a model on an 8-GPU machine: Before running training / inference, setup the configuration file properly

python3 -m torch.distributed.launch --nproc_per_node=8 tools/train_net.py --config-file configs/dla/DLA_34_FPN.yaml --train-dir PATH_TO_TRAIN_DIR --model-suffix MODEL_SUFFIX 

Use the following command to test a model on a single-GPU machine:

python3 tools/test_net.py --config-file configs/dla/DLA_34_FPN.yaml --output-dir PATH_TO_OUTPUT_DIR --model-file PATH_TO_MODEL_FILE --test-dataset DATASET_KEY --set val

Note: If you get an error ModuleNotFoundError: No module named 'siammot' when running in the git root then make sure your PYTHONPATH includes the current directory, which you can add by running: export PYTHONPATH=.:$PYTHONPATH or you can explicitly add the project to the path by replacing the '.' in the export command with the absolute path to the git root.

Multi-gpu testing is going to be supported later.

Version

This is the preliminary version specifically for Airbone Object Tracking (AOT) workshop. The current version only support the motion model being EMM.

  • [Update 06/02/2021] Refactor of configuration file
  • [Update 06/02/2021] Operator patching for amodal inference (needed in MOT17) and model release of MOT17 model
  • [Update 06/02/2021] Support inference based on provided public detection

Stay tuned for more updates

License

This project is licensed under the Apache-2.0 License.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].