All Categories → Machine Learning → attention-is-all-you-need

Top 28 attention-is-all-you-need open source projects

Pytorch Transformer

pytorch implementation of Attention is all you need

✭ 199

python pytorch translation transformer attention-is-all-you-need

Kospeech

Open-Source Toolkit for End-to-End Korean Automatic Speech Recognition.

✭ 190

python pytorch speech-recognition transformer seq2seq asr end-to-end attention-is-all-you-need

Linear Attention Recurrent Neural Network

A recurrent attention module consisting of an LSTM cell which can query its own past cell states by the means of windowed multi-head attention. The formulas are derived from the BN-LSTM and the Transformer Network. The LARNN cell with attention can be easily used inside a loop on the cell state, just like any other RNN. (LARNN)

✭ 119

jupyter-notebook pytorch lstm rnn recurrent-neural-networks attention-mechanism attention-model attention-is-all-you-need

Njunmt Pytorch

✭ 79

python pytorch neural-machine-translation nmt attention-is-all-you-need

Transformers without tears

Transformers without Tears: Improving the Normalization of Self-Attention

✭ 80

python pytorch transformer machine-translation attention-is-all-you-need

Machine Translation

✭ 51

python pytorch transformer seq2seq machine-translation sequence-to-sequence attention-is-all-you-need

Sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

✭ 990

python deep-learning machine-learning deep-neural-networks translation transformer attention-mechanism mxnet seq2seq machine-translation neural-machine-translation encoder-decoder sequence-to-sequence attention-model gluon attention-is-all-you-need

Witwicky

Witwicky: An implementation of Transformer in PyTorch.

✭ 21

python pytorch transformer machine-translation attention-is-all-you-need

Bert language understanding

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

✭ 933

python nlp text-classification transfer-learning question-answering language-model fasttext attention-is-all-you-need

Attention Is All You Need Keras

A Keras+TensorFlow Implementation of the Transformer: Attention Is All You Need

✭ 628

python deep-learning keras keras-tensorflow attention-is-all-you-need

Awesome Fast Attention

list of efficient attention modules

✭ 627

python awesome transformer attention attention-is-all-you-need

Attention Is All You Need Pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".

✭ 6,070

python shell deep-learning pytorch nlp natural-language-processing attention attention-is-all-you-need

Speech Transformer

A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.

✭ 565

python pytorch transformer attention asr end-to-end attention-is-all-you-need

Textclassificationbenchmark

A Benchmark of Text Classification in PyTorch

✭ 534

python pytorch cnn lstm benchmark text-classification quantum crnn attention-is-all-you-need rcnn

Nmt Keras

Neural Machine Translation with Keras

✭ 501

python deep-learning machine-learning tensorflow keras transformer attention-mechanism theano machine-translation neural-machine-translation sequence-to-sequence attention-model gru nmt attention-is-all-you-need

Pytorch Original Transformer

My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing otherwise seemingly hard concepts. Currently included IWSLT pretrained models.

✭ 411

python jupyter-notebook deep-learning pytorch deeplearning jupyter transformer attention-mechanism attention attention-is-all-you-need

Transformer

A TensorFlow Implementation of the Transformer: Attention Is All You Need

✭ 3,646

python perl shell translation transformer attention-mechanism implementation attention-is-all-you-need

Dab

Data Augmentation by Backtranslation (DAB) ヽ( •_-)ᕗ

✭ 294

language jupyter-notebook deep-learning machine-learning deep-neural-networks transformer google-cloud nlp-machine-learning data-augmentation attention-is-all-you-need

Transformer

A Pytorch Implementation of "Attention is All You Need" and "Weighted Transformer Network for Machine Translation"