YerevaNN / Dynamic Memory Networks In Theano
Licence: mit
Implementation of Dynamic memory networks by Kumar et al. http://arxiv.org/abs/1506.07285
Stars: β 334
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Dynamic Memory Networks In Theano
Repo 2016
R, Python and Mathematica Codes in Machine Learning, Deep Learning, Artificial Intelligence, NLP and Geolocation
Stars: β 103 (-69.16%)
Mutual labels: natural-language-processing, theano
Clause
π θ倩ζΊε¨δΊΊοΌθͺηΆθ―θ¨η解οΌθ―δΉη解
Stars: β 323 (-3.29%)
Mutual labels: natural-language-processing
Nlp101
NLP 101: a resource repository for Deep Learning and Natural Language Processing
Stars: β 305 (-8.68%)
Mutual labels: natural-language-processing
Awesome Arabic
A curated list of awesome projects and dev/design resources for supporting Arabic computational needs.
Stars: β 309 (-7.49%)
Mutual labels: natural-language-processing
Gcn Over Pruned Trees
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (authors' PyTorch implementation)
Stars: β 312 (-6.59%)
Mutual labels: natural-language-processing
Graphbrain
Language, Knowledge, Cognition
Stars: β 294 (-11.98%)
Mutual labels: natural-language-processing
Adam qas
ADAM - A Question Answering System. Inspired from IBM Watson
Stars: β 330 (-1.2%)
Mutual labels: natural-language-processing
Ai Deadlines
β° AI conference deadline countdowns
Stars: β 3,852 (+1053.29%)
Mutual labels: natural-language-processing
Deep srl
Code and pre-trained model for: Deep Semantic Role Labeling: What Works and What's Next
Stars: β 309 (-7.49%)
Mutual labels: theano
Biosentvec
BioWordVec & BioSentVec: pre-trained embeddings for biomedical words and sentences
Stars: β 308 (-7.78%)
Mutual labels: natural-language-processing
Zhihu
This repo contains the source code in my personal column (https://zhuanlan.zhihu.com/zhaoyeyu), implemented using Python 3.6. Including Natural Language Processing and Computer Vision projects, such as text generation, machine translation, deep convolution GAN and other actual combat code.
Stars: β 3,307 (+890.12%)
Mutual labels: natural-language-processing
Displacy
π₯ displaCy.js: An open-source NLP visualiser for the modern web
Stars: β 311 (-6.89%)
Mutual labels: natural-language-processing
Nlprule
A fast, low-resource Natural Language Processing and Text Correction library written in Rust.
Stars: β 309 (-7.49%)
Mutual labels: natural-language-processing
Chakin
Simple downloader for pre-trained word vectors
Stars: β 323 (-3.29%)
Mutual labels: natural-language-processing
Nlp
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang
Stars: β 304 (-8.98%)
Mutual labels: natural-language-processing
Trankit
Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing
Stars: β 311 (-6.89%)
Mutual labels: natural-language-processing
Aspect Based Sentiment Analysis
A paper list for aspect based sentiment analysis.
Stars: β 311 (-6.89%)
Mutual labels: natural-language-processing
Nndial
NNDial is an open source toolkit for building end-to-end trainable task-oriented dialogue models. It is released by Tsung-Hsien (Shawn) Wen from Cambridge Dialogue Systems Group under Apache License 2.0.
Stars: β 332 (-0.6%)
Mutual labels: natural-language-processing
Learning
The data is the future of oil, digging the potential value of the data is very meaningful. This library records my road of machine learning study.
Stars: β 330 (-1.2%)
Mutual labels: theano
Dynamic memory networks in Theano
The aim of this repository is to implement Dynamic memory networks as described in the paper by Kumar et al. and to experiment with its various extensions.
Pretrained models on bAbI tasks can be tested online.
We will cover the process in a series of blog posts.
- The first post describes the details of the basic architecture and presents our first results on bAbI tasks v1.2.
-
The second post describes our second model called
dmn_smoothand introduces our playground for bAbI tasks.
Repository contents
| file | description |
|---|---|
main.py |
the main entry point to train and test available network architectures on bAbI-like tasks |
dmn_basic.py |
our baseline implementation. It is as close to the original as we could understand the paper, except the number of steps in the main memory GRU is fixed. Attention module uses T.abs_ function as a distance between two vectors which causes gradients to become NaN randomly. The results reported in this blog post are based on this network |
dmn_smooth.py |
uses the square of the Euclidean distance instead of abs in the attention module. Training is very stable. Performance on bAbI is slightly better |
dmn_batch.py |
dmn_smooth with minibatch training support. The batch size cannot be set to 1 because of the Theano bug
|
dmn_qa_draft.py |
draft version of a DMN designed for answering multiple choice questions |
utils.py |
tools for working with bAbI tasks and GloVe vectors |
nn_utils.py |
helper functions on top of Theano and Lasagne |
fetch_babi_data.sh |
shell script to fetch bAbI tasks (adapted from MemN2N) |
fetch_glove_data.sh |
shell script to fetch GloVe vectors (by 5vision) |
server/ |
contains Flask-based restful api server |
Usage
This implementation is based on Theano and Lasagne. One way to install them is:
pip install -r https://raw.githubusercontent.com/Lasagne/Lasagne/master/requirements.txt
pip install https://github.com/Lasagne/Lasagne/archive/master.zip
The following bash scripts will download bAbI tasks and GloVe vectors.
./fetch_babi_data.sh
./fetch_glove_data.sh
Use main.py to train a network:
python main.py --network dmn_basic --babi_id 1
The states of the network will be saved in states/ folder.
There is one pretrained state on the 1st bAbI task. It should give 100% accuracy on the test set:
python main.py --network dmn_basic --mode test --babi_id 1 --load_state states/dmn_basic.mh5.n40.babi1.epoch4.test0.00033.state
Server
If you want to start a server which will return the predication for bAbi tasks, you should do the following:
- Generate UI files as described in YerevaNN/dmn-ui
- Copy the UI files to
server/ui - Run the server
cd server && python api.py
If have Docker installed, you can pull our Docker image with ready DMN server.
docker pull yerevann/docker
docker run --name dmn_1 -it --rm -p 5000:5000 yerevann/dmn
Roadmap
- Mini-batch training (done, 08/02/2016)
- Web interface (done, 08/23/2016)
- Visualization of episodic memory module (done, 08/23/2016)
- Regularization (work in progress, L2 doesn't help at all, dropout and batch normalization help a little)
- Support for multiple-choice questions (work in progress)
- Evaluation on more complex datasets
- Import some ideas from Neural Reasoner
License
The MIT License (MIT) Copyright (c) 2016 YerevaNN
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].