pandeykartikey / Hierarchical Attention Network
Implementation of Hierarchical Attention Networks in PyTorch
Stars: ✭ 120
Projects that are alternatives of or similar to Hierarchical Attention Network
Glove As A Tensorflow Embedding Layer
Taking a pretrained GloVe model, and using it as a TensorFlow embedding weight layer **inside the GPU**. Therefore, you only need to send the index of the words through the GPU data transfer bus, reducing data transfer overhead.
Stars: ✭ 85 (-29.17%)
Mutual labels: jupyter-notebook, word2vec, glove
Servenet
Service Classification based on Service Description
Stars: ✭ 21 (-82.5%)
Mutual labels: jupyter-notebook, word2vec, glove
Word2vec
訓練中文詞向量 Word2vec, Word2vec was created by a team of researchers led by Tomas Mikolov at Google.
Stars: ✭ 48 (-60%)
Mutual labels: jupyter-notebook, word2vec
Gdax Orderbook Ml
Application of machine learning to the Coinbase (GDAX) orderbook
Stars: ✭ 60 (-50%)
Mutual labels: jupyter-notebook, gru
Vectorsinsearch
Dice.com repo to accompany the dice.com 'Vectors in Search' talk by Simon Hughes, from the Activate 2018 search conference, and the 'Searching with Vectors' talk from Haystack 2019 (US). Builds upon my conceptual search and semantic search work from 2015
Stars: ✭ 71 (-40.83%)
Mutual labels: word2vec, glove
Word2vec Russian Novels
Inspired by word2vec-pride-vis the replacement of words of Russian most valuable novels text with closest word2vec model words. By Boris Orekhov
Stars: ✭ 39 (-67.5%)
Mutual labels: jupyter-notebook, word2vec
Deeplearning Nlp Models
A small, interpretable codebase containing the re-implementation of a few "deep" NLP models in PyTorch. Colab notebooks to run with GPUs. Models: word2vec, CNNs, transformer, gpt.
Stars: ✭ 64 (-46.67%)
Mutual labels: jupyter-notebook, word2vec
Nlp In Practice
Starter code to solve real world text data problems. Includes: Gensim Word2Vec, phrase embeddings, Text Classification with Logistic Regression, word count with pyspark, simple text preprocessing, pre-trained embeddings and more.
Stars: ✭ 790 (+558.33%)
Mutual labels: jupyter-notebook, word2vec
Magnitude
A fast, efficient universal vector embedding utility package.
Stars: ✭ 1,394 (+1061.67%)
Mutual labels: word2vec, glove
Neural Networks
All about Neural Networks!
Stars: ✭ 34 (-71.67%)
Mutual labels: jupyter-notebook, word2vec
Rnn Notebooks
RNN(SimpleRNN, LSTM, GRU) Tensorflow2.0 & Keras Notebooks (Workshop materials)
Stars: ✭ 48 (-60%)
Mutual labels: jupyter-notebook, gru
Twitter sentiment analysis word2vec convnet
Twitter Sentiment Analysis with Gensim Word2Vec and Keras Convolutional Network
Stars: ✭ 24 (-80%)
Mutual labels: jupyter-notebook, word2vec
Repo 2017
Python codes in Machine Learning, NLP, Deep Learning and Reinforcement Learning with Keras and Theano
Stars: ✭ 1,123 (+835.83%)
Mutual labels: word2vec, glove
Ngram2vec
Four word embedding models implemented in Python. Supporting arbitrary context features
Stars: ✭ 703 (+485.83%)
Mutual labels: word2vec, glove
Text2vec
Fast vectorization, topic modeling, distances and GloVe word embeddings in R.
Stars: ✭ 715 (+495.83%)
Mutual labels: word2vec, glove
Awesome Embedding Models
A curated list of awesome embedding models tutorials, projects and communities.
Stars: ✭ 1,486 (+1138.33%)
Mutual labels: jupyter-notebook, word2vec
Textclf
TextClf ：基于Pytorch/Sklearn的文本分类框架，包括逻辑回归、SVM、TextCNN、TextRNN、TextRCNN、DRNN、DPCNN、Bert等多种模型，通过简单配置即可完成数据处理、模型训练、测试等过程。
Stars: ✭ 105 (-12.5%)
Mutual labels: word2vec, glove
Hierarchical Attention Networks for Document Classification
We know that documents have a hierarchical structure, words combine to form sentences and sentences combine to form documents. We can try to learn that structure or we can input this hierarchical structure into the model and see if it improves the performance of existing models. This paper exploits that structure to build a classification model.
This is a (close) implementation of the model in PyTorch.
Keypoints
- The network uses Bidirectional GRU to capture the contextual information about a word.
- There are two layers of attention, one Word level, and another Sentence level.
- It uses word2vec for word embeddings.
- Negative Log Likelihood is used as the loss function.
- The dataset was divided in the ratio 8:1:1 for training, validation, and test respectively.
Note: If you are using NLLLoss from pytorch make sure to use the log_softmax function from the functional class and not softmax
Notebook
The notebook contains was trained on yelp dataset taken from here.
The best accuracy that I got was around ~ 64.6%. This dataset has only 10000 samples and 29 classes. Here is the training loss for the dataset.
Here is the training accuracy for the process.
Here is the validation accuracy for the process.
Attachments
You can find the word2vec model trained on this dataset here and the trained weights of the HAN model here
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].