All Projects → Disiok → Poetry Seq2seq

Disiok / Poetry Seq2seq

Licence: mit
Chinese Poetry Generation

Projects that are alternatives of or similar to Poetry Seq2seq

Pytorch Seq2seq
Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.
Stars: ✭ 3,418 (+2049.69%)
Mutual labels:  jupyter-notebook, lstm, rnn, seq2seq
Linear Attention Recurrent Neural Network
A recurrent attention module consisting of an LSTM cell which can query its own past cell states by the means of windowed multi-head attention. The formulas are derived from the BN-LSTM and the Transformer Network. The LARNN cell with attention can be easily used inside a loop on the cell state, just like any other RNN. (LARNN)
Stars: ✭ 119 (-25.16%)
Mutual labels:  jupyter-notebook, lstm, rnn, attention-mechanism
Natural Language Processing With Tensorflow
Natural Language Processing with TensorFlow, published by Packt
Stars: ✭ 222 (+39.62%)
Mutual labels:  jupyter-notebook, lstm, rnn, seq2seq
Chinese Chatbot
中文聊天机器人,基于10万组对白训练而成,采用注意力机制,对一般问题都会生成一个有意义的答复。已上传模型,可直接运行,跑不起来直播吃键盘。
Stars: ✭ 124 (-22.01%)
Mutual labels:  jupyter-notebook, lstm, rnn, seq2seq
Rnn Notebooks
RNN(SimpleRNN, LSTM, GRU) Tensorflow2.0 & Keras Notebooks (Workshop materials)
Stars: ✭ 48 (-69.81%)
Mutual labels:  jupyter-notebook, lstm, rnn
Neural Networks
All about Neural Networks!
Stars: ✭ 34 (-78.62%)
Mutual labels:  jupyter-notebook, lstm, rnn
Seq2seq chatbot new
基于seq2seq模型的简单对话系统的tf实现,具有embedding、attention、beam_search等功能,数据集是Cornell Movie Dialogs
Stars: ✭ 144 (-9.43%)
Mutual labels:  beam-search, attention-mechanism, seq2seq
Pytorch Pos Tagging
A tutorial on how to implement models for part-of-speech tagging using PyTorch and TorchText.
Stars: ✭ 96 (-39.62%)
Mutual labels:  jupyter-notebook, lstm, rnn
Video Classification
Tutorial for video classification/ action recognition using 3D CNN/ CNN+RNN on UCF101
Stars: ✭ 543 (+241.51%)
Mutual labels:  jupyter-notebook, lstm, rnn
Machine Learning
My Attempt(s) In The World Of ML/DL....
Stars: ✭ 78 (-50.94%)
Mutual labels:  jupyter-notebook, lstm, rnn
Awesome Speech Recognition Speech Synthesis Papers
Automatic Speech Recognition (ASR), Speaker Verification, Speech Synthesis, Text-to-Speech (TTS), Language Modelling, Singing Voice Synthesis (SVS), Voice Conversion (VC)
Stars: ✭ 2,085 (+1211.32%)
Mutual labels:  rnn, attention-mechanism, seq2seq
Lstm Sentiment Analysis
Sentiment Analysis with LSTMs in Tensorflow
Stars: ✭ 886 (+457.23%)
Mutual labels:  jupyter-notebook, lstm, rnn
Stockpriceprediction
Stock Price Prediction using Machine Learning Techniques
Stars: ✭ 700 (+340.25%)
Mutual labels:  jupyter-notebook, lstm, rnn
Bitcoin Price Prediction Using Lstm
Bitcoin price Prediction ( Time Series ) using LSTM Recurrent neural network
Stars: ✭ 67 (-57.86%)
Mutual labels:  jupyter-notebook, lstm, rnn
Telemanom
A framework for using LSTMs to detect anomalies in multivariate time series data. Includes spacecraft anomaly data and experiments from the Mars Science Laboratory and SMAP missions.
Stars: ✭ 589 (+270.44%)
Mutual labels:  jupyter-notebook, lstm, rnn
Lstm chem
Implementation of the paper - Generative Recurrent Networks for De Novo Drug Design.
Stars: ✭ 87 (-45.28%)
Mutual labels:  jupyter-notebook, lstm, rnn
Ml Ai Experiments
All my experiments with AI and ML
Stars: ✭ 107 (-32.7%)
Mutual labels:  jupyter-notebook, lstm, rnn
Kaggle Web Traffic
1st place solution
Stars: ✭ 1,641 (+932.08%)
Mutual labels:  jupyter-notebook, rnn, seq2seq
Text summurization abstractive methods
Multiple implementations for abstractive text summurization , using google colab
Stars: ✭ 359 (+125.79%)
Mutual labels:  jupyter-notebook, rnn, seq2seq
Easy Deep Learning With Keras
Keras tutorial for beginners (using TF backend)
Stars: ✭ 367 (+130.82%)
Mutual labels:  jupyter-notebook, lstm, rnn

Chinese Poetry Generation

This project aims to implement and improve upon the classical Chinese poetry generation system proposed in "Chinese Poetry Generation with Planning based Neural Network".

Generated Sample

Sample generated Chinese poetry

Dependencies

Python 2.7
TensorFlow 1.2.1
Jieba 0.38
Gensim 2.0.0
pypinyin 0.23

Features

Network:

  • [x] Bidirectional encoder
  • [x] Attention decoder

Training and Predicting:

  • [x] Alignment boosted word2vec
  • [x] Data loading mode: only keywords (no preceding sentences)
  • [x] Data loading mode: reversed
  • [x] Data loading mode: aligned
  • [x] Training mode: ground truth
  • [x] Training mode: scheduled sampling
  • [x] Predicting mode: greedy
  • [x] Predicting mode: sampling
  • [ ] Predicting mode: beam search

Refinement:

  • [x] Output refiner
  • [ ] Reinforcement learning tuner
  • [ ] Iterative polishing

Evaluation:

  • [x] Evaluation: rhyming
  • [x] Evaluation: tonal structure
  • [ ] Evaluation: alignment score
  • [ ] Evaluation: BLEU score

Project Structure

Data
data: directory for raw data, processed data, pre-processed starterkit data, and generated poetry samples
model: directory for saved neural network models
log: directory for training logs
notebooks: directory for exploratory/experimental IPython notebooks
training_scripts: directory for sample scripts used for training several basic models

Code
model.py: graph definition
train.py: training logic
predict.py: prediction logic
plan.py: keyword planning logic
main.py: user interaction program

Data Processing

To prepare training data:

python data_utils.py

Detail
This scrip does the following in order:

  1. Parse corpus
  2. Build vocab
  3. Filter quatrains
  4. Count words
  5. Rank words
  6. Generate training data

Note
The TextRank algorithm may take many hours to run.
Instead, you can choose to interrupt the iterations and stop it early,
when the progress shown in the terminal has remained stationary for a long time.

Then, to generate the word embedding:

python word2vec.py

Alternative
As an alternative, we have also provided pre-processed data in the data/starterkit directory
You may simply perform cp data/starterkit/* data/processed to skip the data processing step

Training

To train the default model:

python train.py

To view the full list of configurable training parameters:

python train.py -h

Note
Thus you should almost always train a new model after modifying any of the parameters.
Models are by default saved to model/. To train a new model, you may either remove the existing model from model/
or specify a new model path during training with python train.py --model_dir :new_model:dir:

Generating

To start the user interation program:

python main.py

Similarly, to view the full list of configurable predicting parameters:

python main.py -h

Note
The program currently does not check that predication parameters matches corresponding training parameters.
User has to ensure, in particular, the data loading modes correspond with the ones used during traing.
(e.g. If training data is reversed and aligned, then prediction input should also be reversed and aligned.
Otherwise, results may range from subtle differences in output to total crash.

Evaluating

To generate sample poems for evaluation:

python generate_samples.py

Detail
This script by default randomly samples 4000 poems from the training data and saves them as human poems. Then it uses entire poems as inputs to the planner, to create keywords for the predictor. The predicted poems are saved as machine poems.

To evaluate the generated poems:

python evaluate.py

Further Reading

Auxiliary

  1. "Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks"
  2. "Sequence-to-Sequence Learning as Beam-Search Optimization"
  3. "Tuning Recurrent Neural Networks with Reinforcement Learning"
  4. "Deep Reinforcement Learning for Dialogue Generation"

Poetry Generation

  1. May 10, 2017: "Flexible and Creative Chinese Poetry Generation Using Neural Memory"
  2. Dec 7, 2016: "Chinese Poetry Generation with Planning based Neural Network"
  3. June 19, 2016: "Can Machine Generate Traditional Chinese Poetry? A Feigenbaum Test"

Acknowledgement

  1. The data processing source code is based on DevinZ1993's implementation.
  2. The neural network implementation is inspired by JayParks's work.
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].