Awesome NVIDIA Isaac Gym

Collect some related resources of NVIDIA Isaac Gym

News

• PhysX 5: NVIDIA PhysX 5 SDK

• [02/07/2022] Isaac Gym Preview 4 (1.3.0) is available

• [23/03/2022] GTC2022: Isaac Gym: The Next Generation — High-performance Reinforcement Learning in Omniverse

• [29/10/2021] Isaac Gym Preview 3 is available

• NVIDIA Isaac Sim on Omniverse Now Available in Open Beta 21/06/2021

• Isaac Gym

• Isaac Gym: End-to-End GPU-Accelerated Reinforcement Learning

Official resource

• OmniIsaacGymEnvs

• Isaac Gym

• Isaac SDK

• Isaac gym forum

• Isaac Sim GTC 2021, sim-to-real

• Isaac Sim video toturials

• Training Your JetBot in NVIDIA Isaac Sim

• Training Your NVIDIA JetBot to Avoid Collisions Using NVIDIA Isaac Sim

• Introducing NVIDIA Isaac Gym: End-to-End Reinforcement Learning for Robotics

• Accelerating Robotics Simulation with NVIDIA Omniverse Isaac Sim

• Developing Robotics Applications in Python with NVIDIA Isaac SDK

• Building an Intelligent Robot Dog with the NVIDIA Isaac SDK

• youtube video NVIDIAOmniverse

GTC

• Isaac Gym and Omniverse: High Performance Reinforcement Learning Evolved [A31118]

• Learning Challenging Tasks For Quadrupedal Robots: From Simulation To Reality [A31308]

• Sim-to-Real in Isaac Sim

• Isaac Gym: End-to-End GPU-Accelerated Reinforcement Learning

• Bridging Sim2Real Gap: Simulation Tuning for Training Deep Learning Robotic Perception Models

• Reinforcement Learning and Intralogistics: Soft Actor Critic for Maples Navigation in Warehouses

• Building Robotics Applications Using NVIDIA Isaac SDK

• NVIDIA Isaac SIM — Amazing Robot Models and Tasks Simulated in Isaac Sim 2020.1

• Building Robotics Applications Using NVIDIA Isaac SDK

• Sim-to-Real in Isaac Sim

• Omniverse View 2021.2 - Application Tour

• ISAAC SIM Introduction and Live Demo

• NVIDIA on-demand ISAAC SIM

Papers

Manipulation

• RLAfford: Official Implementation of "RLAfford: End-to-end Affordance Learning with Reinforcement Learning" ICRA 2023.

• mvp: Masked Visual Pre-training for Robotics

• [RSS2022] Factory: Fast contact for robotic assembly: paper, project, code

• [SIGGRAPH2022] ASE: Large-scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters: paper, project, code

• [arxiv2021] Data-Driven Operational Space Control for Adaptative and Robust Robot Manipulation: project, paper, code

• [RSS2021@DO-Sim] DO-Sim: Workshop on Deformable Object Simulation in Robotics: link

• [ICRA2021] Causal Reasoning in Simulationfor Structure and Transfer Learning of Robot Manipulation Policies: paper, project

• [RSS2021 DO-Sim workshop] DefGraspSim: Simulation-based grasping of 3D deformable objects: paper, project, video, code

• [arxiv2021] Search-Based Task Planning with Learned Skill Effect Models for Lifelong Robotic Manipulation: paper, project

• [2021] Deformation-Aware Data-Driven Grasp Synthesis: paper

• [2021] Transferring Dexterous Manipulation from GPU Simulation to a Remote Real-World TriFinger: project, paper, code

• [ICRA2021] In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation: paper, project

• [IROS2021] Reactive Long Horizon Task Execution via Visual Skill and Precondition Models: paper, video

• [CoRL2021] STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulation: paper, project, code

• [ICRA2021] Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections: paper

• [2021] DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulation: paper

• [RSS2021_VLRR] A Simple Method for Complex In-Hand Manipulation: paper, project

Localization

• [CoRL2021] Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning: paper, openreview, code, project

• [ICRA2021] Dynamics Randomization Revisited:A Case Study for Quadrupedal Locomotion: project, paper, video

• [2021] GLiDE: Generalizable Quadrupedal Locomotion in Diverse Environments with a Centroidal Model: project, paper

• [CoRL2020] Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion: paper, video, project, blog

• [RAL2021] Learning a State Representation and Navigation in Cluttered and Dynamic Environments: paper

• [CoRL2020] Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation: paper, project

Others

• [arxiv2021] BayesSimIG: Scalable Parameter Inference for Adaptive Domain Randomization with Isaac Gym: paper, code

• [2021] Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation: paper, video, project

• [NeurIPS2021] Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning: project, paper, openreview

• [RSS2020] Learning Active Task-Oriented Exploration Policies for Bridging the Sim-to-Real Gap: paper, project

• [ICRA2019] Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience: paper, video, project

• [CoRL2018] GPU-Accelerated Robotics Simulation for Distributed Reinforcement Learning: paper, project

RL library

These RL libraries can support the training with Isaac Gym.

• VRKitchen2.0-IndoorKit: Omniverse IndoorKit Extension

• rl_games: rl algorithms with isaac gym

• ElegantRL

• skrl, paper

Related GitHub Repos

• IsaacGymEnvs

• isaacgym_hammering

• isaacgym-utils: Developed by the CMU Intelligent Autonomous Manipulation Lab

• isaacgym_sandbox

• thormang3-gogoro-PPO: Steering-based control of a two-wheeled vehicle using RL-PPO and NVIDIA Isaac Gym

• dvrk_IssacGym, link

• dvrk-shape_servoing

• codebase_thesis

• Bittle_URDF

• SceneCollisionNet

• TD3 applied to the Bittle robot in Isaac Sim

• Isaac-ManipulaRL Deep Reinforcement Learning Framework for Manipulator based on NVIDIA's Isaac-gym, Additional add SAC2019 and Reinforcement Learning from Demonstration Algorithm.

• legged_gym_isaac

• minimal-isaac-gym

• DexterousHands: This is a library that provides dual dexterous hand manipulation tasks through Isaac Gym.

• Safe Multi-Agent Isaac Gym Benchmark: Safe Multi-Agent Isaac Gym benchmark for safe multi-agent reinforcement learning research.

• Bez_IsaacGym: Isaac Gym Reinforcement Learning Environments for humanoid robot Bez

• Bimanual_offlineRL

• isaac_rover_mars_gym

• isaac_rover_2.0

• Rapid Locomotion

• walk-these-ways: Go1 Sim-to-Real Locomotion Starter Kit

• walk-these-ways: A1 Sim-to-Real Locomotion Starter Kit

• shifu: Lightweight Isaac Gym Environment Builder for Any Robot

• Dofbot Reacher: Dofbot Reacher Reinforcement Learning Sim2Real Environment for Omniverse Isaac Gym/Sim

• UR10Reacher: UR10 Reacher Reinforcement Learning Sim2Real Environment for Omniverse Isaac Gym/Sim

• minimal-isaac-gym: A Minimal Example of Isaac Gym with DQN and PPO.

• TimeChamber: A Massively Parallel Large Scale Self-Play Framework.

• Rofunc: The Full Process Python Package for Robot Learning from Demonstration.

• rl-mpc-locomotion: Deep RL for MPC control of Quadruped Robot Locomotion.

Tutorials & Videos

RSS2021 Workshop (https://sites.google.com/view/isaacgym/home)

• Isaac Gym Part 1: Introduction and Getting Started
• Isaac Gym Part 2: Environments, Training, and Tips
• Isaac Gym Part 3A: Academic Labs - University of Toronto
• Isaac Gym Part 3B: Academic Labs - IMLab
• Isaac Gym Part 3C: Academic Labs - Stanford University)
• Isaac Gym Part 3D: Academic Labs - Soft-Body Simulation
• Isaac Gym Part 3E: Academic Labs - Eth Zurich
• Isaac Gym Part 4: New Frontiers in End-to-End GPU Accelerated Reinforcement Learning

Videos

• How to Import Your Robot Into Isaac Sim in NVIDIA Omniverse
• Youtube NVIDIA Omniverse
• Basic Demo of the NVIDIA Isaac Simulator (Part 1)
• Basic Demo of the NVIDIA Isaac Simulator (Part 2)
• Introduction and Live Demo in Isaac Sim - Community Stream
• From Point Clouds to Material Graphs: Explore the Latest in Omniverse Create 2021.3
• Robot Autonomy with the Digital Twin in Isaac Sim
• Can we simulate a real robot? A journey through trying to find a high quality physics simulator for a robot dog/quadruped (using the Petoi Bittle in this case).
• Teaching Robots to Walk w/ Reinforcement Learning Robot sim adventure video part two, covering my attempts to get some reinforcement learning to work with the Bittle robot in the Isaac sim.
• Robot Dog Learns to Walk - Bittle Reinforcement Learning p.3 Further progress with using reinforcement learning to train robot dogs/quadrupeds to walk

Blogs

• A brief introduction to Nvidia Omniverse