lettergram / Sentence Classification
Licence: other
Sentence Classifications with Neural Networks
Stars: ✭ 177
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Sentence Classification
Pytorch Sentiment Analysis
Tutorials on getting started with PyTorch and TorchText for sentiment analysis.
Stars: ✭ 3,209 (+1712.99%)
Mutual labels: cnn, rnn, fasttext
Cnn lstm for text classify
CNN, LSTM, NBOW, fasttext 中文文本分类
Stars: ✭ 90 (-49.15%)
Mutual labels: cnn, rnn
Hyperopt Keras Cnn Cifar 100
Auto-optimizing a neural net (and its architecture) on the CIFAR-100 dataset. Could be easily transferred to another dataset or another classification task.
Stars: ✭ 95 (-46.33%)
Mutual labels: cnn, hyperparameter-tuning
Eeg Dl
A Deep Learning library for EEG Tasks (Signals) Classification, based on TensorFlow.
Stars: ✭ 165 (-6.78%)
Mutual labels: cnn, rnn
Deepseqslam
The Official Deep Learning Framework for Route-based Place Recognition
Stars: ✭ 49 (-72.32%)
Mutual labels: cnn, rnn
Nlp overview
Overview of Modern Deep Learning Techniques Applied to Natural Language Processing
Stars: ✭ 1,104 (+523.73%)
Mutual labels: cnn, rnn
Deep Music Genre Classification
🎵 Using Deep Learning to Categorize Music as Time Progresses Through Spectrogram Analysis
Stars: ✭ 23 (-87.01%)
Mutual labels: cnn, rnn
Mnist Classification
Pytorch、Scikit-learn实现多种分类方法，包括逻辑回归（Logistic Regression）、多层感知机（MLP）、支持向量机（SVM）、K近邻（KNN）、CNN、RNN，极简代码适合新手小白入门，附英文实验报告（ACM模板）
Stars: ✭ 109 (-38.42%)
Mutual labels: cnn, rnn
Codesearchnet
Datasets, tools, and benchmarks for representation learning of code.
Stars: ✭ 1,378 (+678.53%)
Mutual labels: cnn, rnn
Rnn Theano
使用Theano实现的一些RNN代码，包括最基本的RNN，LSTM，以及部分Attention模型，如论文MLSTM等
Stars: ✭ 31 (-82.49%)
Mutual labels: cnn, rnn
Sleepeegnet
SleepEEGNet: Automated Sleep Stage Scoring with Sequence to Sequence Deep Learning Approach
Stars: ✭ 89 (-49.72%)
Mutual labels: cnn, rnn
Deepfakes video classification
Deepfakes Video classification via CNN, LSTM, C3D and triplets
Stars: ✭ 24 (-86.44%)
Mutual labels: cnn, rnn
Pytorch Pos Tagging
A tutorial on how to implement models for part-of-speech tagging using PyTorch and TorchText.
Stars: ✭ 96 (-45.76%)
Mutual labels: cnn, rnn
Text Classification Demos
Neural models for Text Classification in Tensorflow, such as cnn, dpcnn, fasttext, bert ...
Stars: ✭ 144 (-18.64%)
Mutual labels: cnn, fasttext
Tensorflow cookbook
Code for Tensorflow Machine Learning Cookbook
Stars: ✭ 5,984 (+3280.79%)
Mutual labels: cnn, rnn
Eda nlp
Data augmentation for NLP, presented at EMNLP 2019
Stars: ✭ 902 (+409.6%)
Mutual labels: cnn, rnn
Captcharecognition
End-to-end variable length Captcha recognition using CNN+RNN+Attention/CTC (pytorch implementation). 端到端的不定长验证码识别
Stars: ✭ 97 (-45.2%)
Mutual labels: cnn, rnn
Sentence Classification
The goal of this project is to classify sentences, based on type:
- Statement (Declarative Sentence)
- Question (Interrogative Sentence)
- Exclamation (Exclamatory Sentence)
- Command (Imperative Sentence)
Each of the above broad sentence categories can be expanded and can be made more indepth. The way these networks and scripts are designed it should be possible expand to classify other sentence types, provided the data is provided.
This was developed for applications at Metacortex and is accompanied by a guide on building practical/applied neural networks on austingwalters.com.
Please, feel free to add PRs to update, improve, and use freely!
To Install
- Install CUDA and CuDNN if you have a GPU (on your system of choice)
- Install requirements (on python 3, python 2.x will not work)
pip3 install -r requirements.txt --user
To execute:
Pretrained model:
python3 sentence_cnn_save.py models/cnn
To build your own model:
python3 sentence_cnn_save.py models/<model name>
The models/ will load any pretrained model with said name, or retrain it.
See supplemental material for full guide.
Supplemental Material
This repository was created in conjunction with a guide titled Neural Networks to Production, From an Engineer.
Below is the guides table of contents:
- Acquiring & formatting data for deep learning applications
- Word embedding and data splitting
- Bag-of-words to classify sentence types (Dictionary)
- Classify sentences via a multilayer perceptron (MLP)
- Classify sentences via a recurrent neural network (LSTM)
- Convolutional neural networks to classify sentences (CNN)
- FastText for sentence classification (FastText)
- Hyperparameter tuning for sentence classification
Additional, (more complex models) are available in the advanced_modeling directory. Eventually, posts should come out of them.
Dataset
The dataset is created from parsing out the SQuAD dataset and combining it with the SPAADIA dataset.
The samples in the dataset:
- Command 1111
- Statement 80167
- Question 131001
Note: Questions in this case are only one sentence, statements are a single sentence or more. They are classified correctly, but don't include sentences prior to questions.
Results
With the above, we are able to get the following accuracy:
| Model | Accuracy | Train Speed | Classification Speed |
|---|---|---|---|
| Dict | 85% | Fastest | Fastest |
| CNN | 97.80% | Fast (185 μs/step) | Very Fast (35 μs/step) |
| CNN (2-layer) | 99.33% | Fast (210 μs/step) | Very Fast (42 μs/step) |
| MLP | 95.5% | Very Fast (60 μs/step) | Very Fast (42 μs/step) |
| FastText (1-gram) | 94.40% | Fast (83 μs/step) | Very Fast (26 μs/step) |
| FastText (2-gram) | 95.59% | Fast (196 μs/step) | Very Fast (26 μs/step) |
| RNN (LSTM) | 98.49% | Very Slow (7000 μs/step) | Very Slow (1000 μs/step) |
| RNN (GRU) | 99.73% | Very Slow (2000 μs/step) | Very Slow (1000 μs/step) |
| CNN + LSTM | 99.55% | Very Slow (3000 μs/step) | Very Slow (722 μs/step) |
| CNN + GRU | 99.82% | Very Slow (2000 μs/step) | Very Slow (591 μs/step) |
| CNN + MLP | 99.75% | Slow (1000 μs/step) | Fast (97 μs/step) |
With some hyperparameter tuning:
| Model | Accuracy | Train Speed | Classification Speed |
|---|---|---|---|
| Dict | 85% | Fastest | Fastest |
| CNN | 99.40% | Fast (200 μs/step) | Very Fast (26 μs/step) |
| CNN (2-layer) | 99.33% | Fast (210 μs/step) | Very Fast (42 μs/step) |
| MLP | 95.5% | Very Fast (60 μs/step) | Very Fast (42 μs/step) |
| FastText (1-gram) | 94.40% | Fast (117 μs/step) | Very Fast (26 μs/step) |
| FastText (2-gram) | 95.59% | Fast (196 μs/step) | Very Fast (26 μs/step) |
| RNN (LSTM) | 98.49% | Very Slow (7000 μs/step) | Very Slow (1000 μs/step) |
| RNN (GRU) | 99.73% | Very Slow (2000 μs/step) | Very Slow (1000 μs/step) |
| CNN + LSTM | 99.55% | Very Slow (3000 μs/step) | Very Slow (722 μs/step) |
| CNN + GRU | 99.82% | Very Slow (2000 μs/step) | Very Slow (340 μs/step) |
| CNN + MLP | 99.75% | Slow (1000 μs/step) | Fast (97 μs/step) |
Computer Configuration:
- GTX 1080
- 32 Gb RAM
- 8x 3.6 Ghz cores (AMD)
- Arch Linux, up to date on 12/16/2018
CNN Hyperparameter tuning
| Accuracy | Speed | Batch Size | Embedding Dims | Filters | Kernel | Hidden Dims | Epochs |
|---|---|---|---|---|---|---|---|
| 99.40% | 26 μs/step | 64 | 75 | 100 | 5 | 350 | 7 |
| 99.36% | 40 μs/step | 64 | 50 | 250 | 10 | 150 | 5 |
| 99.33% | 25 μs/step | 64 | 75 | 75 | 5 | 350 | 5 |
| 99.31% | 59 μs/step | 64 | 100 | 350 | 5 | 300 | 3 |
| 99.29% | 25 μs/step | 64 | 50 | 100 | 7 | 350 | 5 |
| 99.27% | 62 μs/step | 32 | 75 | 350 | 5 | 250 | 3 |
| 99.25% | 25 μs/step | 64 | 75 | 100 | 3 | 350 | 5 |
| 99.25% | 25 μs/step | 64 | 50 | 100 | 7 | 250 | 3 |
| 99.24% | 53 μs/step | 64 | 75 | 350 | 10 | 250 | 3 |
| 99.23% | 56 μs/step | 64 | 75 | 350 | 10 | 200 | 3 |
| 99.18% | 36 μs/step | 64 | 50 | 250 | 5 | 300 | 5 |
| 99.12% | 52 μs/step | 64 | 75 | 350 | 5 | 250 | 3 |
| 99.11% | 22 μs/step | 64 | 50 | 75 | 5 | 300 | 4 |
| 99.11% | 26 μs/step | 64 | 50 | 100 | 10 | 250 | 3 |
| 99.04% | 62 μs/step | 32 | 75 | 350 | 5 | 350 | 3 |
| 99.00% | 24 μs/step | 64 | 100 | 50 | 5 | 350 | 3 |
| 99.00% | 52 μs/step | 64 | 75 | 350 | 5 | 350 | 3 |
| 99.00% | 40 μs/step | 64 | 75 | 250 | 5 | 350 | 3 |
| 98.84% | 50 μs/step | 64 | 50 | 350 | 10 | 150 | 3 |
| 98.86% | 40 μs/step | 64 | 75 | 250 | 5 | 250 | 3 |
| 98.79% | 26 μs/step | 64 | 50 | 100 | 10 | 150 | 3 |
| 98.76% | 30 μs/step | 128 | 50 | 200 | 3 | 150 | 3 |
| 98.66% | 31 μs/step | 64 | 50 | 150 | 10 | 150 | 3 |
| 98.62% | 45 μs/step | 128 | 100 | 350 | 3 | 250 | 3 |
| 98.17% | 19 μs/step | 64 | 75 | 50 | 3 | 350 | 6 |
| 98.07% | 34 μs/step | 128 | 75 | 250 | 5 | 250 | 3 |
| 98.06% | 45 μs/step | 64 | 75 | 350 | 3 | 250 | 3 |
| 97.53% | 35 μs/step | 128 | 75 | 250 | 5 | 350 | 3 |
| 96.10% | 32 μs/step | 128 | 75 | 250 | 3 | 350 | 3 |
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].