GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → BruceChaun → Nmt

BruceChaun / Nmt

Neural Machine Translation with RNN/ConvS2S/Transoformer

Programming Languages

python
139335 projects - #7 most used programming language

Labels

pytorchneural-machine-translation

Projects that are alternatives of or similar to Nmt

SSAN
How Does Selective Mechanism Improve Self-attention Networks?
Stars: ✭ 18 (+38.46%)
Mutual labels:  neural-machine-translation
Nmtpytorch
Sequence-to-Sequence Framework in PyTorch
Stars: ✭ 392 (+2915.38%)
Mutual labels:  neural-machine-translation
Sentencepiece
Unsupervised text tokenizer for Neural Network-based text generation.
Stars: ✭ 5,540 (+42515.38%)
Mutual labels:  neural-machine-translation
Attention-Visualization
Visualization for simple attention and Google's multi-head attention.
Stars: ✭ 54 (+315.38%)
Mutual labels:  neural-machine-translation
Nmt List
A list of Neural MT implementations
Stars: ✭ 359 (+2661.54%)
Mutual labels:  neural-machine-translation
Joeynmt
Minimalist NMT for educational purposes
Stars: ✭ 420 (+3130.77%)
Mutual labels:  neural-machine-translation
RNNSearch
An implementation of attention-based neural machine translation using Pytorch
Stars: ✭ 43 (+230.77%)
Mutual labels:  neural-machine-translation
Tensorflow Tutorial
TensorFlow and Deep Learning Tutorials
Stars: ✭ 748 (+5653.85%)
Mutual labels:  neural-machine-translation
Tf Seq2seq
Sequence to sequence learning using TensorFlow.
Stars: ✭ 387 (+2876.92%)
Mutual labels:  neural-machine-translation
Opennmt Py
Open Source Neural Machine Translation in PyTorch
Stars: ✭ 5,378 (+41269.23%)
Mutual labels:  neural-machine-translation
pytorch basic nmt
A simple yet strong implementation of neural machine translation in pytorch
Stars: ✭ 66 (+407.69%)
Mutual labels:  neural-machine-translation
Indian ParallelCorpus
Curated list of publicly available parallel corpus for Indian Languages
Stars: ✭ 23 (+76.92%)
Mutual labels:  neural-machine-translation
Nmt Keras
Neural Machine Translation with Keras
Stars: ✭ 501 (+3753.85%)
Mutual labels:  neural-machine-translation
DataAugmentationNMT
Data Augmentation for Neural Machine Translation
Stars: ✭ 26 (+100%)
Mutual labels:  neural-machine-translation
Thumt
An open-source neural machine translation toolkit developed by Tsinghua Natural Language Processing Group
Stars: ✭ 550 (+4130.77%)
Mutual labels:  neural-machine-translation
transformer
Neutron: A pytorch based implementation of Transformer and its variants.
Stars: ✭ 60 (+361.54%)
Mutual labels:  neural-machine-translation
Neuralmonkey
An open-source tool for sequence learning in NLP built on TensorFlow.
Stars: ✭ 400 (+2976.92%)
Mutual labels:  neural-machine-translation
Marian
Fast Neural Machine Translation in C++
Stars: ✭ 777 (+5876.92%)
Mutual labels:  neural-machine-translation
Nematus
Open-Source Neural Machine Translation in Tensorflow
Stars: ✭ 730 (+5515.38%)
Mutual labels:  neural-machine-translation
Seq2seq.pytorch
Sequence-to-Sequence learning using PyTorch
Stars: ✭ 514 (+3853.85%)
Mutual labels:  neural-machine-translation
View All Similar Projects ➔

Neural Machine Translation

This is a PyTorch implementation of neural machine translation.

Requirements

  • PyTorch 0.2/0.3
  • Python 3.5/3.6
  • NumPy
  • NLTK

Models

  1. The recurrent model is [1]. The encoder is a two-layer bidirectinoal GRU and the decoder is a single-layer unidirectional GRU. The attention is computed by a multi-layer perceptron.

  2. The ConvS2S is proposed by [2], which uses entirely convolutional layers to encode and decode.

  3. The Transformer is proposed by [3], which uses multi-head attention and position-wise feed-forward network in both encoder and decoder.

  4. Hashed character n-gram embedding [4], which represents word by its character n-grams. Other procedures are the same.

Usage

  1. Download dataset from WIT3.

  2. Preprocess data

python preprocess.py data/en-de/train.tags.en-de.de data/en-de/train.tags.en-de.en \
    data/en-de/de.train data/en-de/en.train # parse raw data
python preprocess.py data/en-de/de.train data/en-de/de.vocab.p 3 # build source vocab
python preprocess.py data/en-de/en.train data/en-de/en.vocab.p 3 # build target vocab
  1. Train models
python [train_rnn.py|train_cnn.py|train_attn.py] \
    data/en-de de en data/en-de/de.vocab.p data/en-de/en.vocab.p
  1. Evaluate models
python eval.py data/en-de de en [rnn|cnn|attn] \
    data/en-de/de.vocab.p data/en-de/en.vocab.p ckpt/encoder ckpt/decoder

Results

Model Complexity

Model #params(word/n-gram)
RNN 83,941,203 / 64,440,659
ConvS2S 74,432,474 / 54,931,930
Transformer 26,960,896 / 17,771,264

BLEU score

Model beam=1 beam=3 beam=5
RNN(word) 7.3 8.6 8.8
RNN(n-gram) 4.0 4.6 4.8
ConvS2S(word) 14.2 17.2 17.8
ConvS2S(n-gram) 11.0 13.7 14.4
Transformer(word) 4.0 4.1 4.2
Transformer(n-gram) 4.1 4.3 4.2

Inference Speed

We test the inference efficiency with beam size 3 on Tesla K80 with 1 GPU card.

Model #speed(sec)(word/n-gram)
RNN 0.36 / 0.39
ConvS2S 1.64 / 1.68
Transformer 0.49 / 0.51

References

  1. Neural machine translation by jointly learning to align and translate
  2. Convolutional Sequence to Sequence Learning
  3. Attention Is All You Need
  4. Natural language processing with small feed-forward networks
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].