All Projects → tianheyu927 → Pcgrad

tianheyu927 / Pcgrad

Licence: mit
Code for "Gradient Surgery for Multi-Task Learning"

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Pcgrad

Deep reinforcement learning
Resources, papers, tutorials
Stars: ✭ 119 (-17.36%)
Mutual labels:  deep-reinforcement-learning
Muzero Pytorch
Pytorch Implementation of MuZero
Stars: ✭ 129 (-10.42%)
Mutual labels:  deep-reinforcement-learning
Deep Qlearning Agent For Traffic Signal Control
A framework where a deep Q-Learning Reinforcement Learning agent tries to choose the correct traffic light phase at an intersection to maximize traffic efficiency.
Stars: ✭ 136 (-5.56%)
Mutual labels:  deep-reinforcement-learning
Deep Rl Tensorflow
TensorFlow implementation of Deep Reinforcement Learning papers
Stars: ✭ 1,552 (+977.78%)
Mutual labels:  deep-reinforcement-learning
Pytorch Trpo
PyTorch Implementation of Trust Region Policy Optimization (TRPO)
Stars: ✭ 123 (-14.58%)
Mutual labels:  deep-reinforcement-learning
Keras Rl2
Reinforcement learning with tensorflow 2 keras
Stars: ✭ 134 (-6.94%)
Mutual labels:  deep-reinforcement-learning
Hierarchical Actor Critic Hac Pytorch
PyTorch implementation of Hierarchical Actor Critic (HAC) for OpenAI gym environments
Stars: ✭ 116 (-19.44%)
Mutual labels:  deep-reinforcement-learning
D3rlpy
An offline deep reinforcement learning library
Stars: ✭ 139 (-3.47%)
Mutual labels:  deep-reinforcement-learning
A Deep Rl Approach For Sdn Routing Optimization
A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization
Stars: ✭ 125 (-13.19%)
Mutual labels:  deep-reinforcement-learning
Policy Gradient
Minimal Monte Carlo Policy Gradient (REINFORCE) Algorithm Implementation in Keras
Stars: ✭ 135 (-6.25%)
Mutual labels:  deep-reinforcement-learning
Advanced Deep Learning And Reinforcement Learning Deepmind
🎮 Advanced Deep Learning and Reinforcement Learning at UCL & DeepMind | YouTube videos 👉
Stars: ✭ 121 (-15.97%)
Mutual labels:  deep-reinforcement-learning
Rl Quadcopter
Teach a Quadcopter How to Fly!
Stars: ✭ 124 (-13.89%)
Mutual labels:  deep-reinforcement-learning
Ml Agents
Unity Machine Learning Agents Toolkit
Stars: ✭ 12,134 (+8326.39%)
Mutual labels:  deep-reinforcement-learning
Drl Portfolio Management
CSCI 599 deep learning and its applications final project
Stars: ✭ 121 (-15.97%)
Mutual labels:  deep-reinforcement-learning
Machine Learning And Data Science
This is a repository which contains all my work related Machine Learning, AI and Data Science. This includes my graduate projects, machine learning competition codes, algorithm implementations and reading material.
Stars: ✭ 137 (-4.86%)
Mutual labels:  deep-reinforcement-learning
Reinforcementlearning Atarigame
Pytorch LSTM RNN for reinforcement learning to play Atari games from OpenAI Universe. We also use Google Deep Mind's Asynchronous Advantage Actor-Critic (A3C) Algorithm. This is much superior and efficient than DQN and obsoletes it. Can play on many games
Stars: ✭ 118 (-18.06%)
Mutual labels:  deep-reinforcement-learning
Deep Reinforcement Learning In Large Discrete Action Spaces
Implementation of the algorithm in Python 3, TensorFlow and OpenAI Gym
Stars: ✭ 132 (-8.33%)
Mutual labels:  deep-reinforcement-learning
Deep Learning Papers Reading Roadmap
深度学习论文阅读路线图
Stars: ✭ 142 (-1.39%)
Mutual labels:  deep-reinforcement-learning
Finrl Library
FinRL: Financial Reinforcement Learning Framework. Please star. 🔥
Stars: ✭ 3,037 (+2009.03%)
Mutual labels:  deep-reinforcement-learning
Adnet
Attention-guided CNN for image denoising(Neural Networks,2020)
Stars: ✭ 135 (-6.25%)
Mutual labels:  deep-reinforcement-learning

PCGrad

This repository contains code for Gradient Surgery for Multi-Task Learning in TensorFlow v1.0+ (PyTorch implementation forthcoming).

PCGrad is a form of gradient surgery that projects a task’s gradient onto the normal plane of the gradient of any other task that has a conflicting gradient, which achieves substantial gains in efficiency and performance on a range of supervised multi-task learning and multi-task reinforcement learning domains. Moreover, it is model-agnostic and can be combined with previously-proposed multitask architectures for enhanced performance.

Installation

Clone this repo and copy PCGrad_tf.py to your codebase.

Usage

optimizer = PCGrad(tf.train.AdamOptimizer()) # wrap your favorite optimizer
losses = # a list of per-task losses
assert len(losses) == num_tasks
train_op = optimizer.minimize(losses)

Experiments

Our experiments in the paper were based on the following repositories.

CIFAR-100-MTL: RoutingNetworks

NYUv2: MTAN

MultiMNIST: MultiObjectiveOptimization

MT10/MT50/goal-conditioned pushing in MetaWorld: softlearning with modifications (per-task temperature and per-task replay buffers). We will release modified multi-task softlearning code soon.

Reference

Please cite as:

@article{yu2020gradient,
  title={Gradient surgery for multi-task learning},
  author={Yu, Tianhe and Kumar, Saurabh and Gupta, Abhishek and Levine, Sergey and Hausman, Karol and Finn, Chelsea},
  journal={arXiv preprint arXiv:2001.06782},
  year={2020}
}
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].