Neural_Conversation_Models
================================= This implementation contains an extension of seq2seq tutorial for conversation models in Tensorflow:
-
Option to use Beam Search and Beam Size for decoding
-
Currently, it supports
- Simple seq2seq models
- Attention based seq2seq models
-
To get better results use beam search during decoding / inference
Examples of basic model can be found in this paper.
https://arxiv.org/abs/1702.05512
Prerequisites
- Python Python 3.3+
- NLTK
- TensorFlow 0.12.1
Installations
- Mac
virtualenv --no-site-packages -p /usr/local/bin/python3.6 ~/venv-py3
source ~/venv-py3/bin/activate
pip3 install --upgrade \
https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-0.12.1-py3-none-any.whl # for CPU Usage
- Linux with GPU Driver
virtualenv --no-site-packages -p /usr/local/bin/python3.6 ~/venv-py3
source ~/venv-py3/bin/activate
pip3 install --upgrade \ https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp34-cp34m-linux_x86_64.whl
Data
Data accepted is in the tsv format where first component is the context and second is the reply
TSV format Ubuntu Dialog Data can be found here or Git repo.
example :-
- What are you doing ? \t Writing seq2seq model .
Usage
To train a model with Ubuntu dataset:
$ python neural_conversation_model.py --train_dir ubuntu/ --en_vocab_size 60000 --size 512 --data_path ubuntu/train.tsv --dev_data ubuntu/valid.tsv --vocab_path ubuntu/60k_vocan.en --attention
To test an existing model:
$ python neural_conversation_model.py --train_dir ubuntu/ --en_vocab_size 60000 --size 512 --data_path ubuntu/train.tsv --dev_data ubuntu/valid.tsv --vocab_path ubuntu/60k_vocan.en --attention --decode --beam_search --beam_size 25
Todo
- Add other state of art neural models.
- Adding layer normalization( in progress )
https://github.com/pbhatia243/tf-layer-norm
Contact
Parminder Bhatia, [email protected]