Trajectory Prediction with Graph-based Dual-scale Context Fusion

Introduction

(Update Aug 4 2022)

• Add (DDP-based) multi-card training/evaluation/submission code.
• Fix some parameters, update links to the pretrained model.

(NEW) This paper is accepted for publication in the Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022).

This is the project page of the paper

Lu Zhang, Peiliang Li, Jing Chen and Shaojie Shen, "Trajectory Prediction with Graph-based Dual-scale Context Fusion", 2021.

Preprint: Link

Quantitative Results:

Video:

Supplementary Video (Argoverse Tracking dataset):

Demo

• Color scheme: green - predicted trajectories; red - observation & GT trajectories; orange - other agents.

Have a try!

Install dependencies

• Create a new conda env

conda create --name dsp python=3.8
conda activate dsp

• Install PyTorch according to your CUDA version. For RTX 30 series, we recommend CUDA >= 11.1, PyTorch >= 1.8.0.

conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge

• Install Argoverse API, please follow this page.

• Install other dependencies

pip install scikit-image IPython tqdm ipdb tensorboard

• Install PyTorch Scatter, please refer to this page.

Play with pretrained models

Generate a subset of the dataset for testing using the script (it will generate 1k samples):

bash scripts/argo_preproc_small.sh

Download the pretrained model (Google) (Baidu, code: cemi). Move the pretrained model to ./saved_models/, then use the scripts below to get prediction results:

bash scripts/argo_dsp_vis.sh

Since we store each sequence as a single file, the system may raise error OSError: [Erron 24] Too many open files during evaluation and training. You may use the command below to solve this issue:

ulimit -SHn 51200
ulimit -s unlimited

Train from scratch

• Preprocess Argoverse motion forecasting dataset using the script:

bash scripts/argo_preproc_all.sh

Note that the preprocessed dataset is pretty large (~ 90 GB), please reserve enough space for preprocessing.

• Launch (single-card) training using the script:

bash scripts/argo_dsp_train.sh

Citation

If you find this paper useful for your research, please consider citing the following:

@article{zhang2021trajectory,
  title={Trajectory Prediction with Graph-based Dual-scale Context Fusion},
  author={Zhang, Lu and Li, Peiliang and Chen, Jing and Shen, Shaojie},
  journal={arXiv preprint arXiv:2111.01592},
  year={2021}
}

Acknowledgement

We would like to express sincere thanks to the authors of the following tools and packages:

• LaneGCN
• PyTorch Scatter