GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → tinhb92 → rnn_darts_fastai

tinhb92 / rnn_darts_fastai

Licence: other
Implement Differentiable Architecture Search (DARTS) for RNN with fastai

Programming Languages

Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Labels

language-modelingpytorchrecurrent-neural-networksfastaineural-architecture-search

Projects that are alternatives of or similar to rnn darts fastai

Darts
Differentiable architecture search for convolutional and recurrent networks
Stars: ✭ 3,463 (+16390.48%)
Mutual labels:  language-modeling, neural-architecture-search
DeepSegmentor
Sequence Segmentation using Joint RNN and Structured Prediction Models (ICASSP 2017)
Stars: ✭ 17 (-19.05%)
Mutual labels:  recurrent-neural-networks
datastories-semeval2017-task6
Deep-learning model presented in "DataStories at SemEval-2017 Task 6: Siamese LSTM with Attention for Humorous Text Comparison".
Stars: ✭ 20 (-4.76%)
Mutual labels:  recurrent-neural-networks
deepblast
Neural Networks for Protein Sequence Alignment
Stars: ✭ 29 (+38.1%)
Mutual labels:  language-modeling
BossNAS
(ICCV 2021) BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
Stars: ✭ 125 (+495.24%)
Mutual labels:  neural-architecture-search
codeprep
A toolkit for pre-processing large source code corpora
Stars: ✭ 39 (+85.71%)
Mutual labels:  language-modeling
referit3d
Code accompanying our ECCV-2020 paper on 3D Neural Listeners.
Stars: ✭ 59 (+180.95%)
Mutual labels:  language-modeling
course-content-dl
NMA deep learning course
Stars: ✭ 537 (+2457.14%)
Mutual labels:  recurrent-neural-networks
lingua-go
👄 The most accurate natural language detection library for Go, suitable for long and short text alike
Stars: ✭ 684 (+3157.14%)
Mutual labels:  language-modeling
SpeakerDiarization RNN CNN LSTM
Speaker Diarization is the problem of separating speakers in an audio. There could be any number of speakers and final result should state when speaker starts and ends. In this project, we analyze given audio file with 2 channels and 2 speakers (on separate channels).
Stars: ✭ 56 (+166.67%)
Mutual labels:  recurrent-neural-networks
TF-NAS
TF-NAS: Rethinking Three Search Freedoms of Latency-Constrained Differentiable Neural Architecture Search (ECCV2020)
Stars: ✭ 66 (+214.29%)
Mutual labels:  neural-architecture-search
mindware
An efficient open-source AutoML system for automating machine learning lifecycle, including feature engineering, neural architecture search, and hyper-parameter tuning.
Stars: ✭ 34 (+61.9%)
Mutual labels:  neural-architecture-search
entity-network
Tensorflow implementation of "Tracking the World State with Recurrent Entity Networks" [https://arxiv.org/abs/1612.03969] by Henaff, Weston, Szlam, Bordes, and LeCun.
Stars: ✭ 58 (+176.19%)
Mutual labels:  recurrent-neural-networks
deep-learning
Assignmends done for Udacity's Deep Learning MOOC with Vincent Vanhoucke
Stars: ✭ 94 (+347.62%)
Mutual labels:  recurrent-neural-networks
sequence-rnn-py
Sequence analyzing using Recurrent Neural Networks (RNN) based on Keras
Stars: ✭ 28 (+33.33%)
Mutual labels:  recurrent-neural-networks
Water-classifier-fastai
Deploy your Flask web app classifier on Heroku which is written using fastai library.
Stars: ✭ 37 (+76.19%)
Mutual labels:  fastai
Deep-Learning-Experiments-implemented-using-Google-Colab
Colab Compatible FastAI notebooks for NLP and Computer Vision Datasets
Stars: ✭ 16 (-23.81%)
Mutual labels:  fastai
regulatory-prediction
Code and Data to accompany "Dilated Convolutions for Modeling Long-Distance Genomic Dependencies", presented at the ICML 2017 Workshop on Computational Biology
Stars: ✭ 26 (+23.81%)
Mutual labels:  recurrent-neural-networks
Music-Genre-Classification
Genre Classification using Convolutional Neural Networks
Stars: ✭ 27 (+28.57%)
Mutual labels:  fastai
NeuroAI
NeuroAI-UW seminar, a regular weekly seminar for the UW community, organized by NeuroAI Shlizerman Lab.
Stars: ✭ 36 (+71.43%)
Mutual labels:  recurrent-neural-networks
View All Similar Projects ➔

DARTS for RNN with fastai

Language model on Penn Treebank using Differentiable Architecture Search (DARTS) and fastai library.
Blog post.
Based on DARTS: Differentiable Architecture Search by Hanxiao Liu, Karen Simonyan, Yiming Yang.
Check out the original implementation.

Requirements

fastai 1.0.52.dev0 (latest as of 10th April 2019), PyTorch 1.0.

Instructions

  1. Run databunch_nb.ipynb to create databunch
  2. Run train_search_nb.ipynb to search for genotype. ~5 hours on 1 v100 gpu for 1 run.
    RNN search is sensitive to initialization so there should be several runs with different seed
  3. Train that genotype from scratch on train_nb.ipynb. ~1.5 days for 1600 epochs.
  4. Test a model using test_nb.ipynb

Pretrained model

Pretrained model of DARTS_V1 genotype after 600 epochs darts_V1.pth.
Place the file at data/models and run test_nb.ipynb. Loss ~4.22, 68.0 perplexity.
Caveat: I haven't been able to get ~58.0 test perplexity like the original implementation.

fastai dev version installation

git clone https://github.com/fastai/fastai
cd fastai
tools/run-after-git-clone
pip install -e ".[dev]"
git pull
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].