titu1994 / Nested Lstm
Licence: mit
Keras implementation of Nested LSTMs
Stars: ✭ 85
Programming Languages
python
139335 projects - #7 most used programming language
Nested LSTM
Keras implementation of Nested LSTMs from the paper Nested LSTMs
From the paper:
Nested LSTMs add depth to LSTMs via nesting as opposed to stacking. The value of a memory cell in an NLSTM is computed by an LSTM cell, which has its own inner memory cell. Nested LSTMs outperform both stacked and single-layer LSTMs with similar numbers of parameters in our experiments on various character-level language modeling tasks, and the inner memories of an LSTM learn longer term dependencies compared with the higher-level units of a stacked LSTM
Usage
Via Cells
from nested_lstm import NestedLSTMCell
from keras.layers import RNN
ip = Input(shape=(nb_timesteps, input_dim))
x = RNN(NestedLSTMCell(units=64, depth=2))(ip)
...
Via Layer
from nested_lstm import NestedLSTM
ip = Input(shape=(nb_timesteps, input_dim))
x = NestedLSTM(units=64, depth=2)(ip)
...
Difference between Stacked LSTMs and Nested LSTMs (from the paper)
Cell diagram (depth = 2, from the paper)
Acknowledgements
Keras code heavily derived from the Tensorflow implementation - https://github.com/hannw/nlstm
Requirements
- Keras 2.1.3+
- Tensorflow 1.2+ or Theano. CNTK untested.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].