bentrevett / Pytorch Seq2seq

Licence: mit

Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.

Programming Languages

Jupyter Notebook

11667 projects

Projects that are alternatives of or similar to Pytorch Seq2seq

Pytorch Sentiment Analysis

Tutorials on getting started with PyTorch and TorchText for sentiment analysis.

Stars: ✭ 3,209 (-6.11%)

Mutual labels: jupyter-notebook, tutorial, pytorch-tutorial, pytorch-tutorials, lstm, rnn, pytorch-nlp, torchtext

Machine Learning

My Attempt(s) In The World Of ML/DL....

Stars: ✭ 78 (-97.72%)

Mutual labels: jupyter-notebook, tutorial, pytorch-tutorial, lstm, rnn, attention

Pytorch Pos Tagging

A tutorial on how to implement models for part-of-speech tagging using PyTorch and TorchText.

Stars: ✭ 96 (-97.19%)

Mutual labels: jupyter-notebook, tutorial, pytorch-tutorial, pytorch-tutorials, lstm, rnn

Rnn For Joint Nlu

Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling" (https://arxiv.org/abs/1609.01454)

Stars: ✭ 176 (-94.85%)

Mutual labels: jupyter-notebook, lstm, rnn, attention, encoder-decoder

Sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on Apache MXNet

Stars: ✭ 990 (-71.04%)

Mutual labels: seq2seq, neural-machine-translation, sequence-to-sequence, encoder-decoder, transformer

ConvLSTM-PyTorch

ConvLSTM/ConvGRU (Encoder-Decoder) with PyTorch on Moving-MNIST

Stars: ✭ 202 (-94.09%)

Mutual labels: lstm, gru, rnn, encoder-decoder, pytorch-implementation

Chinese Chatbot

中文聊天机器人，基于10万组对白训练而成，采用注意力机制，对一般问题都会生成一个有意义的答复。已上传模型，可直接运行，跑不起来直播吃键盘。

Stars: ✭ 124 (-96.37%)

Mutual labels: jupyter-notebook, lstm, rnn, attention, seq2seq

Load forecasting

Load forcasting on Delhi area electric power load using ARIMA, RNN, LSTM and GRU models

Stars: ✭ 160 (-95.32%)

Mutual labels: jupyter-notebook, lstm, rnn, gru

Poetry Seq2seq

Chinese Poetry Generation

Stars: ✭ 159 (-95.35%)

Mutual labels: jupyter-notebook, lstm, rnn, seq2seq

Eeg Dl

A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.

Stars: ✭ 165 (-95.17%)

Mutual labels: lstm, rnn, gru, transformer

Getting Things Done With Pytorch

Jupyter Notebook tutorials on solving real-world problems with Machine Learning & Deep Learning using PyTorch. Topics: Face detection with Detectron 2, Time Series anomaly detection with LSTM Autoencoders, Object Detection with YOLO v5, Build your first Neural Network, Time Series forecasting for Coronavirus daily cases, Sentiment Analysis with BERT.

Stars: ✭ 738 (-78.41%)

Mutual labels: jupyter-notebook, tutorial, lstm, transformer

Language Translation

Neural machine translator for English2German translation.

Stars: ✭ 82 (-97.6%)

Mutual labels: jupyter-notebook, lstm, neural-machine-translation, sequence-to-sequence

End To End Sequence Labeling Via Bi Directional Lstm Cnns Crf Tutorial

Tutorial for End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF

Stars: ✭ 87 (-97.45%)

Mutual labels: jupyter-notebook, tutorial, pytorch-tutorial, lstm

Nlp Tutorial

Natural Language Processing Tutorial for Deep Learning Researchers

Stars: ✭ 9,895 (+189.5%)

Mutual labels: jupyter-notebook, tutorial, attention, transformer

Rnn Notebooks

RNN(SimpleRNN, LSTM, GRU) Tensorflow2.0 & Keras Notebooks (Workshop materials)

Stars: ✭ 48 (-98.6%)

Mutual labels: jupyter-notebook, lstm, rnn, gru

Nlp Models Tensorflow

Gathers machine learning and Tensorflow deep learning models for NLP problems, 1.13 < Tensorflow < 2.0

Stars: ✭ 1,603 (-53.1%)

Mutual labels: jupyter-notebook, lstm, attention, neural-machine-translation

Pytorch Learners Tutorial

PyTorch tutorial for learners

Stars: ✭ 97 (-97.16%)

Mutual labels: jupyter-notebook, pytorch-tutorial, lstm, rnn

Screenshot To Code

A neural network that transforms a design mock-up into a static website.

Stars: ✭ 13,561 (+296.75%)

Mutual labels: jupyter-notebook, lstm, seq2seq, encoder-decoder

Nmt Keras

Neural Machine Translation with Keras

Stars: ✭ 501 (-85.34%)

Mutual labels: neural-machine-translation, sequence-to-sequence, gru, transformer

Video Classification

Tutorial for video classification/ action recognition using 3D CNN/ CNN+RNN on UCF101

Stars: ✭ 543 (-84.11%)

Mutual labels: jupyter-notebook, pytorch-tutorial, lstm, rnn

View All Similar Projects ➔

PyTorch Seq2Seq

Note: This repo only works with torchtext 0.9 or above which requires PyTorch 1.8 or above. If you are using torchtext 0.8 then please use this branch

This repo contains tutorials covering understanding and implementing sequence-to-sequence (seq2seq) models using PyTorch 1.8, torchtext 0.9 and spaCy 3.0, using Python 3.8.

If you find any mistakes or disagree with any of the explanations, please do not hesitate to submit an issue. I welcome any feedback, positive or negative!

Getting Started

To install PyTorch, see installation instructions on the PyTorch website.

To install torchtext:

pip install torchtext

We'll also make use of spaCy to tokenize our data. To install spaCy, follow the instructions here making sure to install both the English and German models with:

python -m spacy download en_core_web_sm
python -m spacy download de_core_news_sm

Tutorials

1 - Sequence to Sequence Learning with Neural Networks

This first tutorial covers the workflow of a PyTorch with torchtext seq2seq project. We'll cover the basics of seq2seq networks using encoder-decoder models, how to implement these models in PyTorch, and how to use torchtext to do all of the heavy lifting with regards to text processing. The model itself will be based off an implementation of Sequence to Sequence Learning with Neural Networks, which uses multi-layer LSTMs.
2 - Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Now we have the basic workflow covered, this tutorial will focus on improving our results. Building on our knowledge of PyTorch and torchtext gained from the previous tutorial, we'll cover a second second model, which helps with the information compression problem faced by encoder-decoder models. This model will be based off an implementation of Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, which uses GRUs.
3 - Neural Machine Translation by Jointly Learning to Align and Translate

Next, we learn about attention by implementing Neural Machine Translation by Jointly Learning to Align and Translate. This further allievates the information compression problem by allowing the decoder to "look back" at the input sentence by creating context vectors that are weighted sums of the encoder hidden states. The weights for this weighted sum are calculated via an attention mechanism, where the decoder learns to pay attention to the most relevant words in the input sentence.
4 - Packed Padded Sequences, Masking, Inference and BLEU

In this notebook, we will improve the previous model architecture by adding packed padded sequences and masking. These are two methods commonly used in NLP. Packed padded sequences allow us to only process the non-padded elements of our input sentence with our RNN. Masking is used to force the model to ignore certain elements we do not want it to look at, such as attention over padded elements. Together, these give us a small performance boost. We also cover a very basic way of using the model for inference, allowing us to get translations for any sentence we want to give to the model and how we can view the attention values over the source sequence for those translations. Finally, we show how to calculate the BLEU metric from our translations.
5 - Convolutional Sequence to Sequence Learning

We finally move away from RNN based models and implement a fully convolutional model. One of the downsides of RNNs is that they are sequential. That is, before a word is processed by the RNN, all previous words must also be processed. Convolutional models can be fully parallelized, which allow them to be trained much quicker. We will be implementing the Convolutional Sequence to Sequence model, which uses multiple convolutional layers in both the encoder and decoder, with an attention mechanism between them.
6 - Attention Is All You Need

Continuing with the non-RNN based models, we implement the Transformer model from Attention Is All You Need. This model is based soley on attention mechanisms and introduces Multi-Head Attention. The encoder and decoder are made of multiple layers, with each layer consisting of Multi-Head Attention and Positionwise Feedforward sublayers. This model is currently used in many state-of-the-art sequence-to-sequence and transfer learning tasks.

References

Here are some things I looked at while making these tutorials. Some of it may be out of date.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting