GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → markdtw → Meta Learning Lstm Pytorch

markdtw / Meta Learning Lstm Pytorch

pytorch implementation of Optimization as a Model for Few-shot Learning

Programming Languages

python
139335 projects - #7 most used programming language

Labels

pytorchmeta-learning

Projects that are alternatives of or similar to Meta Learning Lstm Pytorch

Learning To Learn By Pytorch
"Learning to learn by gradient descent by gradient descent "by PyTorch -- a simple re-implementation.
Stars: ✭ 31 (-74.38%)
Mutual labels:  meta-learning
Learn2learn
A PyTorch Library for Meta-learning Research
Stars: ✭ 1,193 (+885.95%)
Mutual labels:  meta-learning
Meta Blocks
A modular toolbox for meta-learning research with a focus on speed and reproducibility.
Stars: ✭ 110 (-9.09%)
Mutual labels:  meta-learning
L2p Gnn
Codes and datasets for AAAI-2021 paper "Learning to Pre-train Graph Neural Networks"
Stars: ✭ 48 (-60.33%)
Mutual labels:  meta-learning
Neural Process Family
Code for the Neural Processes website and replication of 4 papers on NPs. Pytorch implementation.
Stars: ✭ 53 (-56.2%)
Mutual labels:  meta-learning
R2d2
[ICLR'19] Meta-learning with differentiable closed-form solvers
Stars: ✭ 96 (-20.66%)
Mutual labels:  meta-learning
Mt Net
Code accompanying the ICML-2018 paper "Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace"
Stars: ✭ 30 (-75.21%)
Mutual labels:  meta-learning
Metarec
PyTorch Implementations For A Series Of Deep Learning-Based Recommendation Models (IN PROGRESS)
Stars: ✭ 120 (-0.83%)
Mutual labels:  meta-learning
Memory Efficient Maml
Memory efficient MAML using gradient checkpointing
Stars: ✭ 60 (-50.41%)
Mutual labels:  meta-learning
What I Have Read
Paper Lists, Notes and Slides, Focus on NLP. For summarization, please refer to https://github.com/xcfcode/Summarization-Papers
Stars: ✭ 110 (-9.09%)
Mutual labels:  meta-learning
Multidigitmnist
Combine multiple MNIST digits to create datasets with 100/1000 classes for few-shot learning/meta-learning
Stars: ✭ 48 (-60.33%)
Mutual labels:  meta-learning
Meta Learning Bert
Meta learning with BERT as a learner
Stars: ✭ 52 (-57.02%)
Mutual labels:  meta-learning
Gnn Meta Attack
Implementation of the paper "Adversarial Attacks on Graph Neural Networks via Meta Learning".
Stars: ✭ 99 (-18.18%)
Mutual labels:  meta-learning
Maml Tf
Tensorflow Implementation of MAML
Stars: ✭ 44 (-63.64%)
Mutual labels:  meta-learning
Fewshotnlp
The source codes of the paper "Improving Few-shot Text Classification via Pretrained Language Representations" and "When Low Resource NLP Meets Unsupervised Language Model: Meta-pretraining Then Meta-learning for Few-shot Text Classification".
Stars: ✭ 115 (-4.96%)
Mutual labels:  meta-learning
Few Shot Text Classification
Few-shot binary text classification with Induction Networks and Word2Vec weights initialization
Stars: ✭ 32 (-73.55%)
Mutual labels:  meta-learning
Pytorch Meta
A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch
Stars: ✭ 1,239 (+923.97%)
Mutual labels:  meta-learning
Metar Cnn
Meta R-CNN : Towards General Solver for Instance-level Low-shot Learning
Stars: ✭ 120 (-0.83%)
Mutual labels:  meta-learning
Boml
Bilevel Optimization Library in Python for Multi-Task and Meta Learning
Stars: ✭ 120 (-0.83%)
Mutual labels:  meta-learning
Maxl
The implementation of "Self-Supervised Generalisation with Meta Auxiliary Learning" [NeurIPS 2019].
Stars: ✭ 101 (-16.53%)
Mutual labels:  meta-learning
View All Similar Projects ➔

Optimization as a Model for Few-shot Learning

Pytorch implementation of Optimization as a Model for Few-shot Learning in ICLR 2017 (Oral)

Model Architecture

Prerequisites

  • python 3+
  • pytorch 0.4+ (developed on 1.0.1 with cuda 9.0)
  • pillow
  • tqdm (a nice progress bar)

Data

  • Mini-Imagenet as described here
    • You can download it from here (~2.7GB, google drive link)

Preparation

  • Make sure Mini-Imagenet is split properly. For example: 
    - data/
  - miniImagenet/
    - train/
      - n01532829/
        - n0153282900000005.jpg
        - ...
      - n01558993/
      - ...
    - val/
      - n01855672/
      - ...
    - test/
      - ...
- main.py
- ...
    • It'd be set if you download and extract Mini-Imagenet from the link above
  • Check out scripts/train_5s_5c.sh, make sure --data-root is properly set

Run

For 5-shot, 5-class training, run

bash scripts/train_5s_5c.sh

Hyper-parameters are referred to the author's repo.

For 5-shot, 5-class evaluation, run (remember to change --resume and --seed arguments)

bash scripts/eval_5s_5c.sh

Notes

seed train episodes val episodes val acc mean val acc std test episodes test acc mean test acc std
719 41000 100 59.08 9.9 100 56.59 8.4
- - - - - 250 57.85 8.6
- - - - - 600 57.76 8.6
53 44000 100 58.04 9.1 100 57.85 7.7
- - - - - 250 57.83 8.3
- - - - - 600 58.14 8.5
  • The results I get from directly running the author's repo can be found here, I have slightly better performance (~5%) but neither results match the number in the paper (60%) (Discussion and help are welcome!).
  • Training with the default settings takes ~2.5 hours on a single Titan Xp while occupying ~2GB GPU memory.
  • The implementation replicates two learners similar to the author's repo:
    • learner_w_grad functions as a regular model, get gradients and loss as inputs to meta learner.
    • learner_wo_grad constructs the graph for meta learner:
      • All the parameters in learner_wo_grad are replaced by cI output by meta learner.
      • nn.Parameters in this model are casted to torch.Tensor to connect the graph to meta learner.
  • Several ways to copy a parameters from meta learner to learner depends on the scenario:
    • copy_flat_params: we only need the parameter values and keep the original grad_fn.
    • transfer_params: we want the values as well as the grad_fn (from cI to learner_wo_grad).
      • .data.copy_ v.s. clone() -> the latter retains all the properties of a tensor including grad_fn.
      • To maintain the batch statistics, load_state_dict is used (from learner_w_grad to learner_wo_grad).

References

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].