XPlane_ML_Reinforcement_Learning_Autopilot
This project documents set up of Reinforcement learning environment for flight control. This will enable each user/learner/student/flight enthusiast to come with their own RL algorithm that can interact with and control in real time an aircraft in a simulation environment. The simulator used is X-Plane (the flight environment). This gives a realistic environment to work with together with parameters to help keep the simulation as close to reality as possible.
Update: Use the leaderboard wiki Page--> (Leaderboard) to add details of your own scenario or (reinforcement learning) algorithm.
GYM-XPLANE ENVIRONMENT INSTALLATION
To install the gym x-palne environment go through the read me file of the folder :
Gym-Xplane-final version
This is the latest version of the actual gym-xplane environment.
The other folders contain additional modules for additional functionalities. The names of such folders would give an idea as to what each of such modules does.
Read the Wiki to get a general idea of the environment and training procedure (Gym-Xplane Wiki).
Requirements
- X Plane (Demo version) (main requirement)
- XPlaneConnect (replace this with different python udp connection file if required)
- RL Algorithm (define your own RL algorithm or use standard baslines. )
- Lua Programming/FlyWithLua (for seamless interaction with XPlane)
General setup
- GYM XPLANE ENVIRONMENT
- MULTI AGENT ENVIRONMENT SET UP (Work in progress)
Agent-Environment Interaction Flow
Example Training Episode
Action Space Parameters
The possible action parameters are given below. The choice of parameters would depend on the task. For example one might decide that the last three parameters are not important for the keep heading task (Gym Xplane Wiki) .
| Action Space Parameter | Action type | Action Value Range |
|---|---|---|
| Latitudinal Stick | Box | [-1,1] |
| Longitudinal Stick | Box | [-1,1] |
| Rudder Pedals | Box | [-1,1] |
| Throttle | Box | [-1/4,1] |
| Gear | Discrete | 0,1 |
| Flaps | Box | [0,1] |
| Speedbrakes | Box | [-0.5,1.5] |
State Space Parameters
The state space parameters are well documented in XPlane Data Ref. The number and configuration of state space parameters would also depend on the task. It is possible to use a derived state spade parameters.
Note that, usually a UDP connection (XPlaneConnect) is required to read any parameter parameter from XPlane. An example of how to read parameter from UDP connection is shown below:
client = xpc.XPlaneConnect() # UDP connector
client.getDREF("sim/flightmodel/position/P")[0][0] # moment P
client.getPOSI() # get the lat, long,altitude, pitch, roll, heading, gear
A typical state space parameter configuration is shown below:
| State Space Parameter | State type | State Value Range |
|---|---|---|
| velocity_x | Box | [0,120] |
| velocity_y | Box | [0,120] |
| delta_heading | Box | [-300,300] |
The range of each parameter value would also depend on the configuration or simulation task.
client (UDP connector) already has a function getPOSI that helps read the latitude, longitude, altitude, pitch, roll, heading. Other parameters could be added by using the client getDREF function. Note that the string "sim/flightmodel/position/P" is gotten from the Xplane Dataref referenced earlier (XPlane Data Ref)
Flight Dashboard
To learn more about this go to : FlightDataVizDashBoard
TO DO
- Use XVFB for fake display.
- Docker support (under test)
- Algorithm implementations (high priority) / GPU support to be included
- Real Time flight and data visualization interface.
- Investigate Federated Learning / Thompson Sampling / policy-space response oracles (psro)