jojonki / Gated Convolutional Networks
A PyTorch implementation of : Language Modeling with Gated Convolutional Networks.
Stars: ✭ 88
Programming Languages
python
139335 projects - #7 most used programming language
Gated-Convolutional-Networks
Language Modeling with Gated Convolutional Networks Yann N. Dauphin, Angela Fan, Michael Auli, David Grangier https://arxiv.org/abs/1612.08083
This is a PyTorch implementation of Facebook AI Research Lab's paper: Language Modeling with Gated Convolutional Networks. This paper applies a convolutional approach to language modelling with a novel Gated-CNN model.
Architecture
Requirements
- Download Google 1 Billion Word dataset.
- PyTorch 0.2.0_3
Reference
TODO
- [ ] impl adaptive softmax
- [ ] bug? accuracy didn't increase from epoch#1
- [ ] support variable sentence length using padding
- [ ] Understand padding at the beginning of sentence
- [ ] impl checkpoints
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].