AC-VRNN: Attentive Conditional-VRNN for Multi-Future Trajectory Prediction

This repository contains the PyTorch code for paper:

AC-VRNN: Attentive Conditional-VRNN for Multi-Future Trajectory Prediction
Alessia Bertugli, Simone Calderara, Pasquale Coscia, Lamberto Ballan, Rita Cucchiara

Model architecture

AC-VRNN is new generative model for multi-future trajectory prediction based on Conditional Variational Recurrent Neural Networks (C-VRNNs). Conditioning relies on prior belief maps, representing most likely moving directions and forcing the model to consider the collective agents’ motion. Human interactions are modeled in a structured way with a graph attention mechanism, providing an online attentive hidden state refinement of the recurrent estimation.

Prerequisites

• Python >= 3.8
• PyTorch >= 1.5
• CUDA 10.0

Datasets

1. ETH/UCY DATSETS

A) SGAN/STAGT dataset version.

B) SR_LSTM version (only Biwi Eth annotations are changed).

C) Social Ways version --> to obtain the dataset take Social-Ways data and use dataset_processing/process_sways.py to process the data for this code.

2. SDD

Download TrajNet benchmark, take training data and use dataset_processing/split_sdd.py to process the data for this code.

Belief Maps

To obtain belief maps for each dataset use dataset_processing/heatmap.py. Two stages are required:

1. Generate statistics to compute the coarse of the global grid. They are obtained calling compute_mean_displacement_[dataset_name] function.
2. Generate belief maps for each dataset calling compute_local_heatmaps_[dataset_name].

Training the model

To train AC-VRNN use models/graph/train.py on ETH/UCY A and B giving it the correct paths. Set model='gat'.

To train AC-VRNN use models/graph/train_dsways.py on ETH/UCY C. Set model='gat'.

To train AC-VRNN use models/graph/train_sdd.py on SDD.

Evaluating the model

To evaluate the model call utils/evaluate_model.py setting the correct paths, and load the dataset you want to test.

Cite

If you have any questions, please contact [email protected] or [email protected], or open an issue on this repo.

If you find this repository useful for your research, please cite the following paper:

@article{Bertugli2021-acvrnn,
   title = {AC-VRNN: Attentive Conditional-VRNN for multi-future trajectory prediction},
   journal = {Computer Vision and Image Understanding},
   pages = {103245},
   year = {2021},
   issn = {1077-3142},
   doi = {https://doi.org/10.1016/j.cviu.2021.103245},
   url = {https://www.sciencedirect.com/science/article/pii/S1077314221000898},
   author = {Alessia Bertugli and Simone Calderara and Pasquale Coscia and Lamberto Ballan and Rita Cucchiara},
   keywords = {Trajectory forecasting, Multi-future prediction, Time series, Variational recurrent neural networks, Graph attention networks}
   }