GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → cjiang2 → Vdcnn

cjiang2 / Vdcnn

Implementation of Very Deep Convolutional Neural Network for Text Classification

Programming Languages

python
139335 projects - #7 most used programming language

Labels

tensorflownlpkerasconvolutional-neural-networkstext-classificationkeras-tensorflow

Projects that are alternatives of or similar to Vdcnn

Very Deep Convolutional Networks For Natural Language Processing In Tensorflow
implement the paper" Very Deep Convolutional Networks for Natural Language Processing"(https://arxiv.org/abs/1606.01781 ) in tensorflow
Stars: ✭ 54 (-65.82%)
Mutual labels:  convolutional-neural-networks, text-classification
Sru Deeplearning Workshop
دوره 12 ساعته یادگیری عمیق با چارچوب Keras
Stars: ✭ 66 (-58.23%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Pneumonia Detection From Chest X Ray Images With Deep Learning
Detecting Pneumonia in Chest X-ray Images using Convolutional Neural Network and Pretrained Models
Stars: ✭ 64 (-59.49%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Text classification
all kinds of text classification models and more with deep learning
Stars: ✭ 7,179 (+4443.67%)
Mutual labels:  convolutional-neural-networks, text-classification
Context
ConText v4: Neural networks for text categorization
Stars: ✭ 120 (-24.05%)
Mutual labels:  convolutional-neural-networks, text-classification
Textclassifier
Text classifier for Hierarchical Attention Networks for Document Classification
Stars: ✭ 985 (+523.42%)
Mutual labels:  convolutional-neural-networks, text-classification
Sign Language
Sign Language Recognition for Deaf People
Stars: ✭ 65 (-58.86%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Text and Audio classification with Bert
Text Classification in Turkish Texts with Bert
Stars: ✭ 34 (-78.48%)
Mutual labels:  text-classification, keras-tensorflow
Har Keras Cnn
Human Activity Recognition (HAR) with 1D Convolutional Neural Network in Python and Keras
Stars: ✭ 97 (-38.61%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Text classification
Text Classification Algorithms: A Survey
Stars: ✭ 1,276 (+707.59%)
Mutual labels:  convolutional-neural-networks, text-classification
Music recommender
Music recommender using deep learning with Keras and TensorFlow
Stars: ✭ 528 (+234.18%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Rcnn Text Classification
Tensorflow Implementation of "Recurrent Convolutional Neural Network for Text Classification" (AAAI 2015)
Stars: ✭ 127 (-19.62%)
Mutual labels:  convolutional-neural-networks, text-classification
Multi Class Text Classification Cnn
Classify Kaggle Consumer Finance Complaints into 11 classes. Build the model with CNN (Convolutional Neural Network) and Word Embeddings on Tensorflow.
Stars: ✭ 410 (+159.49%)
Mutual labels:  convolutional-neural-networks, text-classification
Text Classification Keras
📚 Text classification library with Keras
Stars: ✭ 53 (-66.46%)
Mutual labels:  text-classification, keras-tensorflow
Rmdl
RMDL: Random Multimodel Deep Learning for Classification
Stars: ✭ 375 (+137.34%)
Mutual labels:  convolutional-neural-networks, text-classification
Deep Atrous Cnn Sentiment
Deep-Atrous-CNN-Text-Network: End-to-end word level model for sentiment analysis and other text classifications
Stars: ✭ 64 (-59.49%)
Mutual labels:  convolutional-neural-networks, text-classification
Hdltex
HDLTex: Hierarchical Deep Learning for Text Classification
Stars: ✭ 191 (+20.89%)
Mutual labels:  convolutional-neural-networks, text-classification
Cnn Text Classification Tf Chinese
CNN for Chinese Text Classification in Tensorflow
Stars: ✭ 237 (+50%)
Mutual labels:  convolutional-neural-networks, text-classification
Automatic Image Captioning
Generating Captions for images using Deep Learning
Stars: ✭ 84 (-46.84%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
Motionblur Detection By Cnn
Stars: ✭ 126 (-20.25%)
Mutual labels:  convolutional-neural-networks, keras-tensorflow
View All Similar Projects ➔

VDCNN

Tensorflow Implementation of Very Deep Convolutional Neural Network for Text Classification, proposed by Conneau et al.

Archiecture for VDCNN is now correctly re-implemented with Tensorflow 2 and tf.keras support. A simple training interface is implemented following Tensorflow 2 Expert Tutorial. Feel free to contribute additional utilities like TensorBoard support.

Side Note, if you are a newcomer for NLP text classification:

  • Please checkout new SOTA NLP methods like transformers or Bert.

  • Check out PyTorch for MUCH BETTER dynamic graphing and dataset object support.

    • Current VDCNN implementation is also extremely easy to be ported onto PyTorch.

Prerequisites

  • Python3
  • Tensorflow >= 2.0
  • tensorflow-datasets
  • numpy

Datasets

The original paper tests several NLP datasets, including DBPedia, AG's News, Sogou News and etc.

tensorflow-datasets is used to support AG's News dataset.

Downloads of those NLP text classification datasets can be found here (Many thanks to ArdalanM):

Dataset Classes Train samples Test samples source
AG’s News 4 120 000 7 600 link
Sogou News 5 450 000 60 000 link
DBPedia 14 560 000 70 000 link
Yelp Review Polarity 2 560 000 38 000 link
Yelp Review Full 5 650 000 50 000 link
Yahoo! Answers 10 1 400 000 60 000 link
Amazon Review Full 5 3 000 000 650 000 link
Amazon Review Polarity 2 3 600 000 400 000 link

Parameters Setting

The original paper suggests the following details for training:

  • SGD optimizer with lr 1e-2, decay 0.9.
  • 10 - 15 epochs for convergence.
  • He Initialization.

Some additional parameter settings for this repo:

  • Gradient clipping with norm_value of 7.0, to stablize the training.

Skip connections and pooling are correctly implemented now:

  • k-maxpooling.
  • maxpooling with kernel size of 3 and strides 2.
  • conv pooling with K_i convolutional layer.

For dotted skip connections:

  • Identity with zero padding.
  • Conv1D with kernel size of 1.

Please refer to Conneau et al for their methodology and experiment section in more detail.

Experiments

Results are reported as follows: (i) / (ii)

  • (i): Test set accuracy reported by the paper (acc = 100% - error_rate)
  • (ii): Test set accuracy reproduced by this Keras implementation

TODO: Feel free to report your own experimental results in the following format:

Results for "Identity" Shortcut, "k-max" Pooling: | Depth | ag_news | DBPedia | Sogou News | |:---------------:|:-----------------:|:-----------------:|:-----------------:| | 9 layers | 90.17 / xx.xxxx | 98.44 / xx.xxxx | 96.42 / xx.xxxx |
| 17 layers | 90.61 / xx.xxxx | 98.39 / xx.xxxx | 96.49 / xx.xxxx | | 29 layers | 91.33 / xx.xxxx | 98.59 / xx.xxxx | 96.82 / xx.xxxx | | 49 layers | xx.xx / xx.xxxx | xx.xx / xx.xxxx | xx.xx / xx.xxxx |

Reference

Original preprocessing codes and VDCNN Implementation By geduo15

Train Script and data iterator from Convolutional Neural Network for Text Classification

NLP Datasets Gathered by ArdalanM and Others

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].