All Projects → sksq96 → deep-rl-quadcopter

sksq96 / deep-rl-quadcopter

Licence: other
Implementation of Deep Deterministic Policy Gradients (DDPG) to teach a Quadcopter How to Fly!

Programming Languages

HTML
75241 projects
Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to deep-rl-quadcopter

motion-planner-reinforcement-learning
End to end motion planner using Deep Deterministic Policy Gradient (DDPG) in gazebo
Stars: ✭ 99 (+482.35%)
Mutual labels:  deep-reinforcement-learning, ddpg, deep-deterministic-policy-gradient
pytorch-distributed
Ape-X DQN & DDPG with pytorch & tensorboard
Stars: ✭ 98 (+476.47%)
Mutual labels:  deep-reinforcement-learning, ddpg
Pytorch Drl
PyTorch implementations of various Deep Reinforcement Learning (DRL) algorithms for both single agent and multi-agent.
Stars: ✭ 233 (+1270.59%)
Mutual labels:  deep-reinforcement-learning, ddpg
Rainy
☔ Deep RL agents with PyTorch☔
Stars: ✭ 39 (+129.41%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deep Reinforcement Learning Algorithms
31 projects in the framework of Deep Reinforcement Learning algorithms: Q-learning, DQN, PPO, DDPG, TD3, SAC, A2C and others. Each project is provided with a detailed training log.
Stars: ✭ 167 (+882.35%)
Mutual labels:  deep-reinforcement-learning, ddpg
Machine Learning Is All You Need
🔥🌟《Machine Learning 格物志》: ML + DL + RL basic codes and notes by sklearn, PyTorch, TensorFlow, Keras & the most important, from scratch!💪 This repository is ALL You Need!
Stars: ✭ 173 (+917.65%)
Mutual labels:  deep-reinforcement-learning, ddpg
UAV-DDPG
Code for paper "Computation Offloading Optimization for UAV-assisted Mobile Edge Computing: A Deep Deterministic Policy Gradient Approach"
Stars: ✭ 133 (+682.35%)
Mutual labels:  ddpg, deep-deterministic-policy-gradient
Reinforcement Learning
🤖 Implements of Reinforcement Learning algorithms.
Stars: ✭ 104 (+511.76%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deep-Reinforcement-Learning-for-Automated-Stock-Trading-Ensemble-Strategy-ICAIF-2020
Live Trading. Please star.
Stars: ✭ 1,251 (+7258.82%)
Mutual labels:  deep-reinforcement-learning, ddpg
deeprl-continuous-control
Learning Continuous Control in Deep Reinforcement Learning
Stars: ✭ 14 (-17.65%)
Mutual labels:  deep-reinforcement-learning, ddpg
Minimalrl
Implementations of basic RL algorithms with minimal lines of codes! (pytorch based)
Stars: ✭ 2,051 (+11964.71%)
Mutual labels:  deep-reinforcement-learning, ddpg
LWDRLC
Lightweight deep RL Libraray for continuous control.
Stars: ✭ 14 (-17.65%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deep Reinforcement Learning In Large Discrete Action Spaces
Implementation of the algorithm in Python 3, TensorFlow and OpenAI Gym
Stars: ✭ 132 (+676.47%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deeprl
Modularized Implementation of Deep RL Algorithms in PyTorch
Stars: ✭ 2,640 (+15429.41%)
Mutual labels:  deep-reinforcement-learning, ddpg
Easy Rl
强化学习中文教程,在线阅读地址:https://datawhalechina.github.io/easy-rl/
Stars: ✭ 3,004 (+17570.59%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deep-Reinforcement-Learning-With-Python
Master classic RL, deep RL, distributional RL, inverse RL, and more using OpenAI Gym and TensorFlow with extensive Math
Stars: ✭ 222 (+1205.88%)
Mutual labels:  deep-reinforcement-learning, ddpg
Deep Reinforcement Learning For Automated Stock Trading Ensemble Strategy Icaif 2020
Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy. ICAIF 2020. Please star.
Stars: ✭ 518 (+2947.06%)
Mutual labels:  deep-reinforcement-learning, ddpg
Elegantrl
Lightweight, efficient and stable implementations of deep reinforcement learning algorithms using PyTorch.
Stars: ✭ 575 (+3282.35%)
Mutual labels:  deep-reinforcement-learning, ddpg
deep rl acrobot
TensorFlow A2C to solve Acrobot, with synchronized parallel environments
Stars: ✭ 32 (+88.24%)
Mutual labels:  deep-reinforcement-learning, ddpg
ddpg biped
Repository for Planar Bipedal walking robot in Gazebo environment using Deep Deterministic Policy Gradient(DDPG) using TensorFlow.
Stars: ✭ 65 (+282.35%)
Mutual labels:  ddpg, deep-deterministic-policy-gradient

DeepRL Quadcopter Controller

Teach a Quadcopter How to Fly!

In this project, you will design a Deep Reinforcement Learning agent to control several quadcopter flying tasks, including take-off, hover and landing.

Table of Contents

Install

This project uses ROS (Robot Operating System) as the primary communication mechanism between your agent and the simulation. You can either install it on your own machine ("native install"), or use a Udacity-provided Virtual Machine (recommended).

ROS Virtual Machine

Download the compressed VM disk image and unzip it:

  • Compressed VM Disk Image: RoboVM_V2.1.0.zip
  • MD5 checksum: MD5(Ubuntu 64-bit Robo V2.1.0.ova)= 95bfba89fbdac5f2c0a2be2ae186ddbb

You will need a Virtual Machine player to run the VM, such as VMWare or VirtualBox:

  • VMWare: If you use a Windows/Linux system, you can get Workstation Player for free, or if you're on a Mac, you can get a trial of Fusion.
  • VirtualBox: Download and install the appropriate version for your system.

Open your VM player, and then "Open" / "Import" the VM disk image that you just unzipped (the .ova file).

Configure the settings for your VM to allocate at least 2 processors and 4GB of RAM (more the merrier!). Now launch the VM, and follow the on-screen instructions for one-time setup steps.

  • Username: robond
  • Password: robo-nd

To open a terminal in your VM, press Ctrl+Alt+T. If prompted "Do you want to source ROS?", answer y (yes). This is where you will execute your project code.

ROS Native Install

If you choose to install ROS (Robot Operating System) on your own machine, it is recommended that you use Ubuntu 16.04 LTS as your operating system. To install ROS, please follow the instructions here: ROS Installation

Note: This method is not supported by Udacity. If you have trouble performing a native install of ROS, please visit ROS answers or you can try troubleshooting your install with other students in the Udacity Robotics Slack community (robotics.udacity.com) in the #ros channel.

Download

Project Code

On the machine where you have installed ROS (a VM, or your local machine), create a directory named catkin_ws, and inside it create a subdirectory named src. If you're using a VM, you can also share a folder on your file-system between the host and VM. That might make it easier for you to prepare your report and submit your project for review.

Now clone this repository or download it inside the src directory. This is where you will develop your project code. Your folder structure should look like the following (ROS has a fairly complicated build system, as you will see!):

- ~/catkin_ws/
  - src/
    - RL-Quadcopter/
      - quad_controller_rl/
        - ...

The root of this structure (catkin_ws) is a catkin workspace, which you can use to organize and work on all your ROS-based projects (the name catkin_ws is not mandatory - you can change it to anything you want).

Simulator

Download the Udacity Quadcopter Simulator, nicknamed DroneSim, for your host computer OS here.

To start the simulator, simply run the downloaded executable file. You may need to run the simulator after the roslaunch step mentioned below in the Run section, so that it can connect to a running ROS master.

Note: If you are using a Virtual Machine (VM), you cannot run the simulator inside the VM. You have to download and run the simulator for your host operating system and connect it to your VM (see below).

Connecting the Simulator to a VM

If you are running ROS in a VM, there are a couple of steps necessary to make sure it can communicate with the simulator running on your host system. If not using a VM, these steps are not needed.

Enable Networking on VM

  • VMWare: The default setting should work. To verify, with the VM runnning, go to the Virtual Machine menu > Network Adapter. NAT should be selected.
  • VirtualBox:
    1. In the VirtualBox Manager, go to Global Tools (top-right corner) > Host Network Manager.
    2. Create a new Host-only Network. You can leave the default settings, e.g. Name = "vboxnet0", Ipv4 Address/Mask = "192.168.56.1/24", and DHCP Server enabled.
    3. Switch back to Machine Tools, and with your VM selected, open its Settings.
    4. Go to the Network tab, change "Attached to" (network type) to "Host-only Adapter", and pick "vboxnet0" from the "Name" dropdown.
    5. Hit Ok to save, and (re)start the VM.

Obtain IP Addresses for Host and VM

In a terminal on your host computer, run ifconfig. It will list all the network interfaces available, both physical and virtual. There should be one named something like vmnet or vboxnet. Note the IP address (inet or inet addr) mentioned for that interface, e.g. 192.168.56.1. This is your Host IP address.

Do the same inside the VM. Here the interface may have a different name, but the IP address should have a common prefix. Note down the complete IP address, e.g. 192.168.56.101 - this your VM IP address.

Edit Simulator Settings

Inside the simulator's _Data or /Contents folder (on Mac, right-click the app > Show Package Contents), edit ros_settings.txt:

  • Set vm-ip to the VM IP address and set vm-override to true.
  • Set host-ip to the Host IP address and set host-override to true.

The host and/or VM's IP address can change when it is restarted. If you are experiencing connectivity problems, be sure to check that the actual IP addresses match what you have in ros_settings.txt.

Develop

Starter code is provided in quad_controller_rl/ with all the Python modules (.py files) under the src/quad_controller_rl/ package, and the main project notebook under notebooks/. Take a look at the files there, but you do not have to make any changes to the code at this point. Complete the following two steps first (Build and Run), to ensure your ROS installation is working correctly.

Build

To prepare your code to run with ROS, you will first need to build it. This compiles and links different modules ("ROS nodes") needed for the project. Fortunately, you should only need to do this once, since changes to Python scripts don't need recompilation.

  • Go to your catkin workspace (catkin_ws/):
$ cd ~/catkin_ws/
  • Build ROS nodes:
$ catkin_make
  • Enable command-line tab-completion and some other useful ROS utilities:
$ source devel/setup.bash

Run

To run your project, start ROS with the rl_controller.launch file:

$ roslaunch quad_controller_rl rl_controller.launch

You should see a few messages on the terminal as different nodes get spun up. Now you can run the simulator, which is a separate Unity application (note that you must start ROS first, and then run the simulator). Once the simulator initializes itself, you should start seeing additional messages in your ROS terminal, indicating a new episode starting every few seconds. The quadcopter in the simulation should show its blades running as it gets control inputs from the agent, and it should reset at the beginning of each episode.

Tip: If you get tired of this two-step startup process, edit the quad_controller_rl/scripts/drone_sim script and enter a command that runs the simulator application. It will then be launched automatically with ROS!

Note: If you want to learn more about how ROS works and how you can use it for robotics applications, you may enroll in Udacity's Robotics Nanodegree Program and can take the ROS Essentials module.

Implement

Once you have made sure ROS and the simulator are running without any errors, and that they can communicate with each other, try modifying the code in agents/policy_search.py - this is a sample agent that runs by default (e.g. add a print statement). Every time you make a change, you will need to stop the simulator (press Esc with the simulator window active), and shutdown ROS (press Ctrl+C in the terminal). Save your change, and roslaunch again.

Now you should be ready to start coding! Open the project notebook for further instructions (assuming you are in your catkin workspace):

$ jupyter notebook src/RL-Quadcopter/quad_controller_rl/notebooks/RL-Quadcopter.ipynb

Submit

Complete the required sections in the notebook. Once done, save/export the notebook as a PDF or HTML file. This will serve as your project report.

If you are enrolled in a Udacity Nanodegree program, you can submit your completed project for review. Prepare and upload a zip file that includes the following:

  • RL-Quadcopter.ipynb: Project notebook with all sections completed.
  • RL-Quadcopter.pdf / RL-Quadcopter.html: PDF or HTML report (which can be an export of the notebook).
  • quad_controller_rl/: Python package with your code in the following sub-packages:
    • tasks/: Your implementation of each task required for this project.
    • agents/: Agents you've implemented for the corresponding tasks.
    • Any other supporting code or other files used by your project.
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].