mindspore-nlp-tutorial

mindspore-nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using MindSpore. This repository is migrated from nlp-tutorial. Most of the models in NLP were migrated from Pytorch version with less than 100 lines of code.(except comments or blank lines)

• Notice: All models are implemented in GPU version, but not test on Ascend platform.

Curriculum - (Example Purpose)

1. Basic Embedding Model

• 1-1. NNLM(Neural Network Language Model) - Predict Next Word
  • Paper - A Neural Probabilistic Language Model(2003)
• 1-2. Word2Vec(Skip-gram) - Embedding Words and Show Graph
  • Paper - Distributed Representations of Words and Phrases and their Compositionality(2013)

2. CNN(Convolutional Neural Network)

• 2-1. TextCNN - Binary Sentiment Classification
  • Paper - Convolutional Neural Networks for Sentence Classification(2014)

3. RNN(Recurrent Neural Network)

• 3-1. TextRNN - Predict Next Step
  • Paper - Finding Structure in Time(1990)
• 3-2. TextLSTM - Autocomplete
  • Paper - LONG SHORT-TERM MEMORY(1997)
• 3-3. Bi-LSTM - Predict Next Word in Long Sentence

4. Attention Mechanism

• 4-1. Seq2Seq - Change Word
  • Paper - Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation(2014)

5. Model based on Transformer

• 5-1. The Transformer - Translate
  • Paper - Attention Is All You Need(2017)

Dependencies

• Python 3.7.5
• MindSpore 1.1.1(GPU version)
• Pytorch 1.7.1(for comparation)

Author

• Yufeng Lyu
• Author Email : [email protected]
• Acknowledgements to graykode who opensource the Pytorch and Tensorflow version.