GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → HaniehP → PersianNER

HaniehP / PersianNER

Licence: other
Named-Entity Recognition in Persian Language

Labels

natural-language-processingword-embeddingsannotationsembeddingsnamed-entitiesdatasetnamed-entity-recognitionpersian-languagepersian-nlppersian-ner

Projects that are alternatives of or similar to PersianNER

Embeddingsviz
Visualize word embeddings of a vocabulary in TensorBoard, including the neighbors
Stars: ✭ 40 (-16.67%)
Mutual labels:  word-embeddings, embeddings
Magnitude
A fast, efficient universal vector embedding utility package.
Stars: ✭ 1,394 (+2804.17%)
Mutual labels:  word-embeddings, embeddings
Chinese Word Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量
Stars: ✭ 9,548 (+19791.67%)
Mutual labels:  word-embeddings, embeddings
sentiment-analysis-of-tweets-in-russian
Sentiment analysis of tweets in Russian using Convolutional Neural Networks (CNN) with Word2Vec embeddings.
Stars: ✭ 51 (+6.25%)
Mutual labels:  word-embeddings, embeddings
Persian-Summarization
Statistical and Semantical Text Summarizer in Persian Language
Stars: ✭ 38 (-20.83%)
Mutual labels:  persian-language, persian-nlp
SentimentAnalysis
Sentiment Analysis: Deep Bi-LSTM+attention model
Stars: ✭ 32 (-33.33%)
Mutual labels:  word-embeddings, embeddings
Fastrtext
R wrapper for fastText
Stars: ✭ 103 (+114.58%)
Mutual labels:  word-embeddings, embeddings
Entity Recognition Datasets
A collection of corpora for named entity recognition (NER) and entity recognition tasks. These annotated datasets cover a variety of languages, domains and entity types.
Stars: ✭ 891 (+1756.25%)
Mutual labels:  annotations, named-entity-recognition
Dna2vec
dna2vec: Consistent vector representations of variable-length k-mers
Stars: ✭ 117 (+143.75%)
Mutual labels:  word-embeddings, embeddings
Flair
A very simple framework for state-of-the-art Natural Language Processing (NLP)
Stars: ✭ 11,065 (+22952.08%)
Mutual labels:  word-embeddings, named-entity-recognition
lda2vec
Mixing Dirichlet Topic Models and Word Embeddings to Make lda2vec from this paper https://arxiv.org/abs/1605.02019
Stars: ✭ 27 (-43.75%)
Mutual labels:  word-embeddings, embeddings
Datastories Semeval2017 Task4
Deep-learning model presented in "DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis".
Stars: ✭ 184 (+283.33%)
Mutual labels:  word-embeddings, embeddings
word2vec-tsne
Google News and Leo Tolstoy: Visualizing Word2Vec Word Embeddings using t-SNE.
Stars: ✭ 59 (+22.92%)
Mutual labels:  word-embeddings, embeddings
Named Entity Recognition With Bidirectional Lstm Cnns
Named-Entity-Recognition-with-Bidirectional-LSTM-CNNs
Stars: ✭ 283 (+489.58%)
Mutual labels:  word-embeddings, named-entity-recognition
datastories-semeval2017-task6
Deep-learning model presented in "DataStories at SemEval-2017 Task 6: Siamese LSTM with Attention for Humorous Text Comparison".
Stars: ✭ 20 (-58.33%)
Mutual labels:  word-embeddings, embeddings
Dict2vec
Dict2vec is a framework to learn word embeddings using lexical dictionaries.
Stars: ✭ 91 (+89.58%)
Mutual labels:  word-embeddings, embeddings
Ner Lstm
Named Entity Recognition using multilayered bidirectional LSTM
Stars: ✭ 532 (+1008.33%)
Mutual labels:  embeddings, named-entity-recognition
Yedda
YEDDA: A Lightweight Collaborative Text Span Annotation Tool. Code for ACL 2018 Best Demo Paper Nomination.
Stars: ✭ 704 (+1366.67%)
Mutual labels:  annotations, named-entity-recognition
Danlp
DaNLP is a repository for Natural Language Processing resources for the Danish Language.
Stars: ✭ 111 (+131.25%)
Mutual labels:  word-embeddings, named-entity-recognition
Hash Embeddings
PyTorch implementation of Hash Embeddings (NIPS 2017). Submission to the NIPS Implementation Challenge.
Stars: ✭ 126 (+162.5%)
Mutual labels:  word-embeddings, embeddings
View All Similar Projects ➔

PersianNER

Named-Entity Recognition in Persian Language

ArmanPersoNERCorpus

This is the first manually-annotated Persian named-entity (NE) dataset (ISLRN 399-379-640-828-6). We are releasing it only for academic research use.

The dataset includes 250,015 tokens and 7,682 Persian sentences in total. It is available in 3 folds to be used in turn as training and test sets. Each file contains one token, along with its manually annotated named-entity tag, per line. Each sentence is separated with a newline. The NER tags are in IOB format.

According to the instructions provided to the annotators, NEs are categorized into six classes: person, organization (such as banks, ministries, embassies, teams, nationalities, networks and publishers), location (such as cities, villages, rivers, seas, gulfs, deserts and mountains), facility (such as schools, universities, research centers, airports, railways, bridges, roads, harbors, stations, hospitals, parks, zoos and cinemas), product (such as books, newspapers, TV shows, movies, airplanes, ships, cars, theories, laws, agreements and religions), and event (such as wars, earthquakes, national holidays, festivals and conferences); other are the remaining tokens.

Please cite [1,2] if you use this dataset.

Persian Word Embeddings

GloVe300d, word2vec_cbow300d, word2vec_skipgram300d, and hpca300d are four different word embeddings trained on a sizable collation of unannotated Persian text. They contain a comprehensive Persian dictionary of nearly 50K unique words. The length of the embedding vectors is 300. The use of these embeddings is unrestricted.

Please cite [2] if you use any of these word embeddings in your work.

Reported Scores

Paper CoNLL F1 Score*
[1] 65.13%
[2] 77.45%
https://ieeexplore.ieee.org/abstract/document/8661067 76.79%
https://ieeexplore.ieee.org/abstract/document/8700549 84.23%

*The CoNLL F1 score is a strict version of the standard F1 score where a true positive is scored only if all the tokens of a given named entity are classified correctly (including their B- and I- tags). Conversely, every incorrect B- prediction is counted as a false positive. You may find the conlleval Perl script here https://www.clips.uantwerpen.be/conll2000/chunking/conlleval.txt .

Citations

  1. Hanieh Poostchi, Ehsan Zare Borzeshi, Mohammad Abdous, and Massimo Piccardi, "PersoNER: Persian Named-Entity Recognition," The 26th International Conference on Computational Linguistics (COLING 2016), pages 3381–3389, Osaka, Japan, 2016.

  2. Hanieh Poostchi, Ehsan Zare Borzeshi, and Massimo Piccardi, "BiLSTM-CRF for Persian Named-Entity Recognition; ArmanPersoNERCorpus: the First Entity-Annotated Persian Dataset," The 11th Edition of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan, 7-12 May 2018, ISLRN 399-379-640-828-6, ISLRN 921-509-141-609-6.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].