hav4ik / Hydra
Licence: mit
Multi-Task Learning Framework on PyTorch. State-of-the-art methods are implemented to effectively train models on multiple tasks.
Stars: ✭ 87
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Hydra
Neural Architecture Search
Basic implementation of [Neural Architecture Search with Reinforcement Learning](https://arxiv.org/abs/1611.01578).
Stars: ✭ 352 (+304.6%)
Mutual labels: neural-architecture-search
Slimmable networks
Slimmable Networks, AutoSlim, and Beyond, ICLR 2019, and ICCV 2019
Stars: ✭ 708 (+713.79%)
Mutual labels: neural-architecture-search
Nsganetv2
[ECCV2020] NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Search
Stars: ✭ 52 (-40.23%)
Mutual labels: neural-architecture-search
Fasterseg
[ICLR 2020] "FasterSeg: Searching for Faster Real-time Semantic Segmentation" by Wuyang Chen, Xinyu Gong, Xianming Liu, Qian Zhang, Yuan Li, Zhangyang Wang
Stars: ✭ 438 (+403.45%)
Mutual labels: neural-architecture-search
Paddleslim
PaddleSlim is an open-source library for deep model compression and architecture search.
Stars: ✭ 677 (+678.16%)
Mutual labels: neural-architecture-search
Morph Net
Fast & Simple Resource-Constrained Learning of Deep Network Structure
Stars: ✭ 937 (+977.01%)
Mutual labels: neural-architecture-search
Darts
Differentiable architecture search for convolutional and recurrent networks
Stars: ✭ 3,463 (+3880.46%)
Mutual labels: neural-architecture-search
Tenas
[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang
Stars: ✭ 63 (-27.59%)
Mutual labels: neural-architecture-search
Awesome Automl And Lightweight Models
A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hyperparameter Optimization, 5.) Automated Feature Engineering.
Stars: ✭ 691 (+694.25%)
Mutual labels: neural-architecture-search
Autokeras
AutoML library for deep learning
Stars: ✭ 8,269 (+9404.6%)
Mutual labels: neural-architecture-search
Hpbandster
a distributed Hyperband implementation on Steroids
Stars: ✭ 456 (+424.14%)
Mutual labels: neural-architecture-search
Randwirenn
Implementation of: "Exploring Randomly Wired Neural Networks for Image Recognition"
Stars: ✭ 675 (+675.86%)
Mutual labels: neural-architecture-search
Neural Architecture Search With Rl
Minimal Tensorflow implementation of the paper "Neural Architecture Search With Reinforcement Learning" presented at ICLR 2017
Stars: ✭ 37 (-57.47%)
Mutual labels: neural-architecture-search
Autogan
[ICCV 2019] "AutoGAN: Neural Architecture Search for Generative Adversarial Networks" by Xinyu Gong, Shiyu Chang, Yifan Jiang and Zhangyang Wang
Stars: ✭ 388 (+345.98%)
Mutual labels: neural-architecture-search
Awesome Architecture Search
A curated list of awesome architecture search resources
Stars: ✭ 1,078 (+1139.08%)
Mutual labels: neural-architecture-search
Adanet
Fast and flexible AutoML with learning guarantees.
Stars: ✭ 3,340 (+3739.08%)
Mutual labels: neural-architecture-search
Devol
Genetic neural architecture search with Keras
Stars: ✭ 925 (+963.22%)
Mutual labels: neural-architecture-search
Autodl Projects
Automated deep learning algorithms implemented in PyTorch.
Stars: ✭ 1,187 (+1264.37%)
Mutual labels: neural-architecture-search
Mtlnas
[CVPR 2020] MTL-NAS: Task-Agnostic Neural Architecture Search towards General-Purpose Multi-Task Learning
Stars: ✭ 58 (-33.33%)
Mutual labels: neural-architecture-search
Efficientnas
Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search https://arxiv.org/abs/1807.06906
Stars: ✭ 44 (-49.43%)
Mutual labels: neural-architecture-search
Hydra — a Multi-Task Learning Framework
Hydra is a flexible multi-task learning framework written in PyTorch 1.0. The following multi-objective optimization algorithms are implemented:
- Naive — a separate optimizer for each task
- Gradients averaging — average out the gradients to the network's body
- MGDA — described in the paper Multi-Task Learning as Multi-Objective Optimization (NIPS 2018)
A comprehensive survey on these algorithms (and more) can be found in this blog article.
Installation
-
The code was written on
Python 3.6. Clone this repository:git clone https://github.com/hav4ik/Hydra -
It is recommended to use anaconda for installation of core packages (since
condapackages comes with low-level libraries that can optimize the runtime):conda install pytorch torchvision cudatoolkit=10.0 -c pytorch conda install numpy pandas scikit-learn -
Some of the packages are not available from anaconda, so you can install them using
pip:pip install -r requirements.txt
Getting started
-
Examples of configuration files can be found here. A minimal example is available in starter.sh. Execute it as follows (will train with configurations in configs/toy_experiments/naive.yaml):
./starter.sh naive 50
Coming soon...
- Proper framework documentation and examples.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].