All Projects → Shuijing725 → CrowdNav_DSRNN

Shuijing725 / CrowdNav_DSRNN

Licence: MIT license
[ICRA 2021] Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to CrowdNav DSRNN

go-orca
Golang implementation of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm
Stars: ✭ 40 (-6.98%)
Mutual labels:  collision-avoidance
drift drl
High-speed Autonomous Drifting with Deep Reinforcement Learning
Stars: ✭ 82 (+90.7%)
Mutual labels:  deep-reinforcement-learning
DeepLearningFlappyFrog
Flappy Frog hack using Deep Reinforcement Learning (Deep Q-learning). 暴力膜蛤不可取。
Stars: ✭ 16 (-62.79%)
Mutual labels:  deep-reinforcement-learning
deep-rts
A Real-Time-Strategy game for Deep Learning research
Stars: ✭ 152 (+253.49%)
Mutual labels:  deep-reinforcement-learning
multi view ram
No description or website provided.
Stars: ✭ 26 (-39.53%)
Mutual labels:  deep-reinforcement-learning
Meta-Learning-for-StarCraft-II-Minigames
We reproduced DeepMind's results and implement a meta-learning (MLSH) agent which can generalize across minigames.
Stars: ✭ 26 (-39.53%)
Mutual labels:  deep-reinforcement-learning
decentralized-rl
Decentralized Reinforcment Learning: Global Decision-Making via Local Economic Transactions (ICML 2020)
Stars: ✭ 40 (-6.98%)
Mutual labels:  deep-reinforcement-learning
MP-DQN
Source code for the dissertation: "Multi-Pass Deep Q-Networks for Reinforcement Learning with Parameterised Action Spaces"
Stars: ✭ 99 (+130.23%)
Mutual labels:  deep-reinforcement-learning
open-phri
OpenPHRI, a complete and generic solution for safe physical human-robot interactions
Stars: ✭ 24 (-44.19%)
Mutual labels:  human-robot-interaction
alpha sigma
A pytorch based Gomoku game model. Alpha Zero algorithm based reinforcement Learning and Monte Carlo Tree Search model.
Stars: ✭ 134 (+211.63%)
Mutual labels:  deep-reinforcement-learning
chi
A high-level framework for advanced deep learning with TensorFlow
Stars: ✭ 55 (+27.91%)
Mutual labels:  deep-reinforcement-learning
LWDRLC
Lightweight deep RL Libraray for continuous control.
Stars: ✭ 14 (-67.44%)
Mutual labels:  deep-reinforcement-learning
AI
使用深度强化学习解决视觉跟踪和视觉导航问题
Stars: ✭ 16 (-62.79%)
Mutual labels:  deep-reinforcement-learning
TF RL
Eagerly Experimentable!!!
Stars: ✭ 22 (-48.84%)
Mutual labels:  deep-reinforcement-learning
Carla-ppo
This repository hosts a customized PPO based agent for Carla. The goal of this project is to make it easier to interact with and experiment in Carla with reinforcement learning based agents -- this, by wrapping Carla in a gym like environment that can handle custom reward functions, custom debug output, etc.
Stars: ✭ 122 (+183.72%)
Mutual labels:  deep-reinforcement-learning
mmn
Moore Machine Networks (MMN): Learning Finite-State Representations of Recurrent Policy Networks
Stars: ✭ 39 (-9.3%)
Mutual labels:  deep-reinforcement-learning
imitation learning
PyTorch implementation of some reinforcement learning algorithms: A2C, PPO, Behavioral Cloning from Observation (BCO), GAIL.
Stars: ✭ 93 (+116.28%)
Mutual labels:  deep-reinforcement-learning
neptus
Neptus, Command and Control Infrastructure
Stars: ✭ 54 (+25.58%)
Mutual labels:  human-robot-interaction
code summarization public
source code for 'Improving automatic source code summarization via deep reinforcement learning'
Stars: ✭ 71 (+65.12%)
Mutual labels:  deep-reinforcement-learning
minerva
An out-of-the-box GUI tool for offline deep reinforcement learning
Stars: ✭ 80 (+86.05%)
Mutual labels:  deep-reinforcement-learning

DSRNN_CrowdNav

This repository contains the codes for our paper titled "Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning" in ICRA 2021. For more details, please refer to the project website and arXiv preprint. For experiment demonstrations, please refer to the youtube video.

Abstract

Safe and efficient navigation through human crowds is an essential capability for mobile robots. Previous work on robot crowd navigation assumes that the dynamics of all agents are known and well-defined. In addition, the performance of previous methods deteriorates in partially observable environments and environments with dense crowds. To tackle these problems, we propose decentralized structural-Recurrent Neural Network (DS-RNN), a novel network that reasons about spatial and temporal relationships for robot decision making in crowd navigation. We train our network with model-free deep reinforcement learning without any expert supervision. We demonstrate that our model outperforms previous methods in challenging crowd navigation scenarios. We successfully transfer the policy learned in the simulator to a real-world TurtleBot 2i.

Setup

  1. Install Python3.6 (The code may work with other versions of Python, but 3.6 is highly recommended).
  2. Install the required python package using pip or conda. For pip, use the following command:
pip install -r requirements.txt

For conda, please install each package in requirements.txt into your conda environment manually and follow the instructions on the anaconda website.

  1. Install OpenAI Baselines.
git clone https://github.com/openai/baselines.git
cd baselines
pip install -e .
  1. Install Python-RVO2 library.

Getting started

This repository is organized in three parts:

  • crowd_sim/ folder contains the simulation environment. Details of the simulation framework can be found here.
  • crowd_nav/ folder contains configurations and non-neural network policies
  • pytorchBaselines/ contains the code for the DSRNN network and ppo algorithm.

Below are the instructions for training and testing policies.

Change configurations

  1. Environment configurations and training hyperparameters: modify crowd_nav/configs/config.py
  • For FoV environment (left in the figure below): change the value of robot.FOV in config.py
  • For Group environment (right in the figure below): set sim.group_human to True in config.py

Run the code

  1. Train a policy.
python train.py 
  1. Test policies.
    Please modify the test arguments in the begining of test.py.
    We provide two trained example weights for each type of robot kinematics:
    • Holonomic: data/example_model/checkpoints/27776.pt
    • Unicycle: data/example_model_unicycle/checkpoints/55554.pt
python test.py 
  1. Plot training curve.
python plot.py

(We only tested our code in Ubuntu 16.04 and 18.04 with Python 3.6.)

Learning curves

Learning curves of DS-RNN in 360 degrees FoV environment with 5 humans.

Citation

If you find the code or the paper useful for your research, please cite our paper:

@inproceedings{liu2020decentralized,
  title={Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning},
  author={Liu, Shuijing and Chang, Peixin and Liang, Weihang and Chakraborty, Neeloy and Driggs-Campbell, Katherine},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2021},
  pages={3517-3524}
}

Credits

Other contributors:
Peixin Chang
Neeloy Chakraborty

Part of the code is based on the following repositories:

[1] C. Chen, Y. Liu, S. Kreiss, and A. Alahi, “Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning,” in International Conference on Robotics and Automation (ICRA), 2019, pp. 6015–6022. (Github: https://github.com/vita-epfl/CrowdNav)

[2] I. Kostrikov, “Pytorch implementations of reinforcement learning algorithms,” https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail, 2018.

[3] A. Vemula, K. Muelling, and J. Oh, “Social attention: Modeling attention in human crowds,” in IEEE international Conference on Robotics and Automation (ICRA), 2018, pp. 1–7. (Github: https://github.com/jeanoh/big)

Contact

If you have any questions or find any bugs, please feel free to open an issue or pull request.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].