All Projects → FengZiYjun → CharLM

FengZiYjun / CharLM

Licence: MIT license
Character-aware Neural Language Model implemented by PyTorch

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to CharLM

Char Rnn Chinese
Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch. Based on code of https://github.com/karpathy/char-rnn. Support Chinese and other things.
Stars: ✭ 192 (+500%)
Mutual labels:  language-model
Mead Baseline
Deep-Learning Model Exploration and Development for NLP
Stars: ✭ 238 (+643.75%)
Mutual labels:  language-model
rnn-theano
RNN(LSTM, GRU) in Theano with mini-batch training; character-level language models in Theano
Stars: ✭ 68 (+112.5%)
Mutual labels:  language-model
Protein Sequence Embedding Iclr2019
Source code for "Learning protein sequence embeddings using information from structure" - ICLR 2019
Stars: ✭ 194 (+506.25%)
Mutual labels:  language-model
Xlnet zh
中文预训练XLNet模型: Pre-Trained Chinese XLNet_Large
Stars: ✭ 207 (+546.88%)
Mutual labels:  language-model
PLBART
Official code of our work, Unified Pre-training for Program Understanding and Generation [NAACL 2021].
Stars: ✭ 151 (+371.88%)
Mutual labels:  language-model
Nlp learning
结合python一起学习自然语言处理 (nlp): 语言模型、HMM、PCFG、Word2vec、完形填空式阅读理解任务、朴素贝叶斯分类器、TFIDF、PCA、SVD
Stars: ✭ 188 (+487.5%)
Mutual labels:  language-model
asr24
24-hour Automatic Speech Recognition
Stars: ✭ 27 (-15.62%)
Mutual labels:  language-model
Relational Rnn Pytorch
An implementation of DeepMind's Relational Recurrent Neural Networks in PyTorch.
Stars: ✭ 236 (+637.5%)
Mutual labels:  language-model
pd3f
🏭 PDF text extraction pipeline: self-hosted, local-first, Docker-based
Stars: ✭ 132 (+312.5%)
Mutual labels:  language-model
Lingvo
Lingvo
Stars: ✭ 2,361 (+7278.13%)
Mutual labels:  language-model
Pytorch Nce
The Noise Contrastive Estimation for softmax output written in Pytorch
Stars: ✭ 204 (+537.5%)
Mutual labels:  language-model
TF-NNLM-TK
A toolkit for neural language modeling using Tensorflow including basic models like RNNs and LSTMs as well as more advanced models.
Stars: ✭ 20 (-37.5%)
Mutual labels:  language-model
Gpt Scrolls
A collaborative collection of open-source safe GPT-3 prompts that work well
Stars: ✭ 195 (+509.38%)
Mutual labels:  language-model
Vaaku2Vec
Language Modeling and Text Classification in Malayalam Language using ULMFiT
Stars: ✭ 68 (+112.5%)
Mutual labels:  language-model
Automatic Speech Recognition
🎧 Automatic Speech Recognition: DeepSpeech & Seq2Seq (TensorFlow)
Stars: ✭ 192 (+500%)
Mutual labels:  language-model
Zeroth
Kaldi-based Korean ASR (한국어 음성인식) open-source project
Stars: ✭ 248 (+675%)
Mutual labels:  language-model
calm
Context Aware Language Models
Stars: ✭ 29 (-9.37%)
Mutual labels:  language-model
KB-ALBERT
KB국민은행에서 제공하는 경제/금융 도메인에 특화된 한국어 ALBERT 모델
Stars: ✭ 215 (+571.88%)
Mutual labels:  language-model
COCO-LM
[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
Stars: ✭ 109 (+240.63%)
Mutual labels:  language-model

PyTorch-Character-Aware-Neural-Language-Model

Build Status GitHub license

This is the PyTorch implementation of character-aware neural language model proposed in this paper by Yoon Kim.

Requirements

The code is run and tested with Python 3.5.2 and PyTorch 0.3.1.

HyperParameters

HyperParam value
LSTM batch size 20
LSTM sequence length 35
LSTM hidden units 300
epochs 35
initial learning rate 1.0
character embedding dimension 15

Demo

Train the model with split train/valid/test data.

python train.py

The trained model will saved in cache/net.pkl. Test the model.

python test.py

Best result on test set: PPl=127.2163 cross entropy loss=4.8459

Acknowledgement

This implementation borrowed ideas from

https://github.com/jarfo/kchar

https://github.com/cronos123/Character-Aware-Neural-Language-Models

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].