dengxiuqi / Weibosentiment
基于各种机器学习和深度学习的中文微博情感分析
Stars: ✭ 75
Labels
Projects that are alternatives of or similar to Weibosentiment
Google earth engine notebook
A walkthrough of some Google Earth Engine Features, as well as using the data in TensorFlow
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Smartnoise Samples
Code samples and documentation for SmartNoise differential privacy tools
Stars: ✭ 75 (+0%)
Mutual labels: jupyter-notebook
Squeezenet Neural Style Pytorch
Lightweight implementation of neural style in Pytorch with SqueezeNet
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Nlp Tutorial
Natural Language Processing Tutorial for Deep Learning Researchers
Stars: ✭ 9,895 (+13093.33%)
Mutual labels: jupyter-notebook
Keras Multi Gpu
Multi-GPU data-parallel training in Keras
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Lpproj
Scikit-learn compatible Locality Preserving Projections in Python
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Covid 19
Data analysis and visualizations of daily COVID cases report
Stars: ✭ 75 (+0%)
Mutual labels: jupyter-notebook
Mit 6.s094
MIT-6.S094: Deep Learning for Self-Driving Cars Assignments solutions
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Image retrieval
Image retrieval system demo based on caffe and lsh
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Mml Book.github.io
Companion webpage to the book "Mathematics For Machine Learning"
Stars: ✭ 9,488 (+12550.67%)
Mutual labels: jupyter-notebook
Selective Joint Fine Tuning
Codes and models for the CVPR 2017 spotlight paper "Borrowing Treasures from the Wealthy: Deep Transfer Learning through Selective Joint Fine-tuning".
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Deetctionupperbound
Code for calculating the upper bound AP in object detection
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Ios Coreml Mnist
Real-time Number Recognition using Apple's CoreML 2.0 and MNIST -
Stars: ✭ 74 (-1.33%)
Mutual labels: jupyter-notebook
Mit Deep Learning
Tutorials, assignments, and competitions for MIT Deep Learning related courses.
Stars: ✭ 8,912 (+11782.67%)
Mutual labels: jupyter-notebook
Practicalsessions2019
Materials for the practical sessions at EEML2019
Stars: ✭ 75 (+0%)
Mutual labels: jupyter-notebook
Pytorch Book
PyTorch tutorials and fun projects including neural talk, neural style, poem writing, anime generation (《深度学习框架PyTorch:入门与实战》)
Stars: ✭ 9,546 (+12628%)
Mutual labels: jupyter-notebook
WeiboSentiment
基于各种机器学习和深度学习的中文微博情感分析
语料来源: https://github.com/dengxiuqi/weibo2018
"情感分析"是我本科的毕业设计, 也是我入门并爱上NLP的项目hhh, 把它发出来供大家交流
- 用FastText在语料库上训练并生成词向量, 该任务语料库较小, 用fastText可以增加n-gram特征, 比传统word2vec要好
- 训练集10000条语料, 测试集500条语料
- 分别用SVM, Bayes, DNN, LSTM, Attention+BiLSTM, XGBoost等多种模型搭建并训练
正负情感二分类器
-
SVM
其实不太适合做NLP, 只是当年我还很菜所以选了SVM(尬笑), 出于情怀就贴上来了 -
Bayes
速度快, 效果好。可能是因为该任务语料规模较小,在大规模语料任务上性能会下降,而且磁带模型丢失了语序信息,可拓展性不强(2020.2.13更新) -
DNN
效果不好, 不过现在也很少有直接用DNN做NLP的, 所以这里仅作为从机器学习到深度学习的过渡模型了 -
LSTM
用到了上游训练的FastText词向量, 并且考虑了语序信息, 效果有明显提升 -
Attention+BiLSTM
效果很好, 但相比纯LSTM提升没那么明显,主要是因为该任务相对简单且语料少。迁移至更复杂任务后注意力的强大会越来越明显 -
XGBoost
真是机器学习界的一大杀器, 在这种简单的NLP任务上真是又快又好(2020.6.4更新)
-
- 对不同话题下的100条微博进行简单的舆情分析(正负情感微博比例)
实验结果
各种分类器在测试集上的测试结果
模型 | 准确率 |
---|---|
XGBoost | 0.874 |
Attention+BiLSTM | 0.86 |
Naive Bayes | 0.856 |
LSTM | 0.854 |
SVM | 0.82 |
DNN | 0.81 |
舆情分析
数据来自于2018年6月的微博
主题 | 正面:负面 |
---|---|
特朗普 | 55:44 |
周杰伦 | 88:12 |
好莱坞 | 79:21 |
人工智能 | 79:21 |
毕业 | 78:22 |
工程结构
WeiboSentiment
├── 00.FastText.ipynb `生成FastText词向量`
├── 01.SVM.ipynb `SVM分类器`
├── 02.Bayes.ipynb `朴素贝叶斯`
├── 03.DNN.ipynb `神经网络分类器`
├── 04.LSTM.ipynb `LSTM分类器`
├── 05.Attention+BiLSTM.ipynb `Attention+BiLSTM分类器`
├── 06.XGBoost.ipynb `XGBoost分类器`
├── SentimentAnlysis.ipynb `验证集分析`
├── stopwords.txt `停用词典`
├── utils.py `工具函数`
├── model `各种模型`
│ ├── model_100.txt `维度为100的FastText词向量`
│ ├── attention `Attention+LSTM模型`
│ ├── lstm `LSTM模型`
│ └── nn `神经网络模型`
└── weibo2018 `微博语料数据`
├── topics `未标注情感的不同主题微博语料`
├── train.txt `训练集`
└── test.txt `测试集`
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].