MishaLaskin / Curl
Licence: mit
CURL: Contrastive Unsupervised Representation Learning for Sample-Efficient Reinforcement Learning
Stars: ✭ 346
Programming Languages
python
139335 projects - #7 most used programming language
Labels
Projects that are alternatives of or similar to Curl
Rad
RAD: Reinforcement Learning with Augmented Data
Stars: ✭ 268 (-22.54%)
Mutual labels: reinforcement-learning, deep-neural-networks, deep-reinforcement-learning, deep-learning-algorithms, deep-q-network
Hands On Reinforcement Learning With Python
Master Reinforcement and Deep Reinforcement Learning using OpenAI Gym and TensorFlow
Stars: ✭ 640 (+84.97%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-learning-algorithms, deep-q-network
Awesome Deep Reinforcement Learning
Curated list for Deep Reinforcement Learning (DRL): software frameworks, models, datasets, gyms, baselines...
Stars: ✭ 95 (-72.54%)
Mutual labels: reinforcement-learning, deep-neural-networks, deep-reinforcement-learning
Reinforcement Learning
🤖 Implements of Reinforcement Learning algorithms.
Stars: ✭ 104 (-69.94%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-learning-algorithms
2048 Deep Reinforcement Learning
Trained A Convolutional Neural Network To Play 2048 using Deep-Reinforcement Learning
Stars: ✭ 169 (-51.16%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Deep Q Learning
Minimal Deep Q Learning (DQN & DDQN) implementations in Keras
Stars: ✭ 1,013 (+192.77%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Rl Course Experiments
Stars: ✭ 73 (-78.9%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Gdrl
Grokking Deep Reinforcement Learning
Stars: ✭ 304 (-12.14%)
Mutual labels: gpu, reinforcement-learning, deep-reinforcement-learning
Pytorch Drl
PyTorch implementations of various Deep Reinforcement Learning (DRL) algorithms for both single agent and multi-agent.
Stars: ✭ 233 (-32.66%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Reinforcement Learning
Minimal and Clean Reinforcement Learning Examples
Stars: ✭ 2,863 (+727.46%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Drlkit
A High Level Python Deep Reinforcement Learning library. Great for beginners, prototyping and quickly comparing algorithms
Stars: ✭ 29 (-91.62%)
Mutual labels: gpu, reinforcement-learning, deep-reinforcement-learning
Deeprl Tutorials
Contains high quality implementations of Deep Reinforcement Learning algorithms written in PyTorch
Stars: ✭ 748 (+116.18%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Pysc2 Examples
StarCraft II - pysc2 Deep Reinforcement Learning Examples
Stars: ✭ 722 (+108.67%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Deep Learning Drizzle
Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!!
Stars: ✭ 9,717 (+2708.38%)
Mutual labels: reinforcement-learning, deep-neural-networks, deep-reinforcement-learning
Softlearning
Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.
Stars: ✭ 713 (+106.07%)
Mutual labels: reinforcement-learning, deep-neural-networks, deep-reinforcement-learning
Accel Brain Code
The purpose of this repository is to make prototypes as case study in the context of proof of concept(PoC) and research and development(R&D) that I have written in my website. The main research topics are Auto-Encoders in relation to the representation learning, the statistical machine learning for energy-based models, adversarial generation networks(GANs), Deep Reinforcement Learning such as Deep Q-Networks, semi-supervised learning, and neural network language model for natural language processing.
Stars: ✭ 166 (-52.02%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Trixi
Manage your machine learning experiments with trixi - modular, reproducible, high fashion. An experiment infrastructure optimized for PyTorch, but flexible enough to work for your framework and your tastes.
Stars: ✭ 211 (-39.02%)
Mutual labels: deep-neural-networks, deeplearning, deep-reinforcement-learning
Tensorflow Reinforce
Implementations of Reinforcement Learning Models in Tensorflow
Stars: ✭ 480 (+38.73%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Reinforcementlearning.jl
A reinforcement learning package for Julia
Stars: ✭ 192 (-44.51%)
Mutual labels: reinforcement-learning, deep-reinforcement-learning, deep-q-network
Gbrain
GPU Javascript Library for Machine Learning
Stars: ✭ 48 (-86.13%)
Mutual labels: gpu, reinforcement-learning, deeplearning
CURL: Contrastive Unsupervised Representation Learning for Sample-Efficient Reinforcement Learning
This repository is the official implementation of CURL for the DeepMind control experiments. Atari experiments were done in a separate codebase available here. Our implementation of SAC is based on SAC+AE by Denis Yarats.
Installation
All of the dependencies are in the conda_env.yml file. They can be installed manually or with the following command:
conda env create -f conda_env.yml
Instructions
To train a CURL agent on the cartpole swingup task from image-based observations run bash script/run.sh from the root of this directory. The run.sh file contains the following command, which you can modify to try different environments / hyperparamters.
CUDA_VISIBLE_DEVICES=0 python train.py \
--domain_name cartpole \
--task_name swingup \
--encoder_type pixel \
--action_repeat 8 \
--save_tb --pre_transform_image_size 100 --image_size 84 \
--work_dir ./tmp \
--agent curl_sac --frame_stack 3 \
--seed -1 --critic_lr 1e-3 --actor_lr 1e-3 --eval_freq 10000 --batch_size 128 --num_train_steps 1000000
In your console, you should see printouts that look like:
| train | E: 221 | S: 28000 | D: 18.1 s | R: 785.2634 | BR: 3.8815 | A_LOSS: -305.7328 | CR_LOSS: 190.9854 | CU_LOSS: 0.0000
| train | E: 225 | S: 28500 | D: 18.6 s | R: 832.4937 | BR: 3.9644 | A_LOSS: -308.7789 | CR_LOSS: 126.0638 | CU_LOSS: 0.0000
| train | E: 229 | S: 29000 | D: 18.8 s | R: 683.6702 | BR: 3.7384 | A_LOSS: -311.3941 | CR_LOSS: 140.2573 | CU_LOSS: 0.0000
| train | E: 233 | S: 29500 | D: 19.6 s | R: 838.0947 | BR: 3.7254 | A_LOSS: -316.9415 | CR_LOSS: 136.5304 | CU_LOSS: 0.0000
For reference, the maximum score for cartpole swing up is around 845 pts, so CURL has converged to the optimal score. This takes about an hour of training depending on your GPU.
Log abbreviation mapping:
train - training episode
E - total number of episodes
S - total number of environment steps
D - duration in seconds to train 1 episode
R - mean episode reward
BR - average reward of sampled batch
A_LOSS - average loss of actor
CR_LOSS - average loss of critic
CU_LOSS - average loss of the CURL encoder
All data related to the run is stored in the specified working_dir. To enable model or video saving, use the --save_model or --save_video flags. For all available flags, inspect train.py. To visualize progress with tensorboard run:
tensorboard --logdir log --port 6006
and go to localhost:6006 in your browser. If you're running headlessly, try port forwarding with ssh.
For GPU accelerated rendering, make sure EGL is installed on your machine and set export MUJOCO_GL=egl. For environment troubleshooting issues, see the DeepMind control documentation.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].