All Projects → tuzhucheng → Sentence Similarity

tuzhucheng / Sentence Similarity

Licence: mit
PyTorch implementations of various deep learning models for paraphrase detection, semantic similarity, and textual entailment

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Sentence Similarity

Cdqa
⛔ [NOT MAINTAINED] An End-To-End Closed Domain Question Answering System.
Stars: ✭ 500 (+420.83%)
Mutual labels:  question-answering, natural-language-processing
Knowledge Graphs
A collection of research on knowledge graphs
Stars: ✭ 845 (+780.21%)
Mutual labels:  question-answering, natural-language-processing
Paper Reading
Paper reading list in natural language processing, including dialogue systems and text generation related topics.
Stars: ✭ 508 (+429.17%)
Mutual labels:  question-answering, natural-language-processing
Jack
Jack the Reader
Stars: ✭ 242 (+152.08%)
Mutual labels:  question-answering, natural-language-processing
Cdqa Annotator
⛔ [NOT MAINTAINED] A web-based annotator for closed-domain question answering datasets with SQuAD format.
Stars: ✭ 48 (-50%)
Mutual labels:  question-answering, natural-language-processing
Adam qas
ADAM - A Question Answering System. Inspired from IBM Watson
Stars: ✭ 330 (+243.75%)
Mutual labels:  question-answering, natural-language-processing
Insuranceqa Corpus Zh
🚁 保险行业语料库,聊天机器人
Stars: ✭ 821 (+755.21%)
Mutual labels:  question-answering, natural-language-processing
Clicr
Machine reading comprehension on clinical case reports
Stars: ✭ 123 (+28.13%)
Mutual labels:  question-answering, natural-language-processing
Conversational Ai
Conversational AI Reading Materials
Stars: ✭ 34 (-64.58%)
Mutual labels:  question-answering, natural-language-processing
Acl18 results
Code to reproduce results in our ACL 2018 paper "Did the Model Understand the Question?"
Stars: ✭ 31 (-67.71%)
Mutual labels:  question-answering, natural-language-processing
Cmrc2018
A Span-Extraction Dataset for Chinese Machine Reading Comprehension (CMRC 2018)
Stars: ✭ 238 (+147.92%)
Mutual labels:  question-answering, natural-language-processing
Turkish Bert Nlp Pipeline
Bert-base NLP pipeline for Turkish, Ner, Sentiment Analysis, Question Answering etc.
Stars: ✭ 85 (-11.46%)
Mutual labels:  question-answering, natural-language-processing
Pytorch Question Answering
Important paper implementations for Question Answering using PyTorch
Stars: ✭ 154 (+60.42%)
Mutual labels:  question-answering, natural-language-processing
Gnn4nlp Papers
A list of recent papers about Graph Neural Network methods applied in NLP areas.
Stars: ✭ 405 (+321.88%)
Mutual labels:  question-answering, natural-language-processing
Medquad
Medical Question Answering Dataset of 47,457 QA pairs created from 12 NIH websites
Stars: ✭ 129 (+34.38%)
Mutual labels:  question-answering, natural-language-processing
Chat
基于自然语言理解与机器学习的聊天机器人,支持多用户并发及自定义多轮对话
Stars: ✭ 516 (+437.5%)
Mutual labels:  question-answering, natural-language-processing
Chatbot
Русскоязычный чатбот
Stars: ✭ 106 (+10.42%)
Mutual labels:  question-answering, natural-language-processing
Dynamic Coattention Network Plus
Dynamic Coattention Network Plus (DCN+) TensorFlow implementation. Question answering using Deep NLP.
Stars: ✭ 117 (+21.88%)
Mutual labels:  question-answering, natural-language-processing
Spago
Self-contained Machine Learning and Natural Language Processing library in Go
Stars: ✭ 854 (+789.58%)
Mutual labels:  question-answering, natural-language-processing
Bidaf Keras
Bidirectional Attention Flow for Machine Comprehension implemented in Keras 2
Stars: ✭ 60 (-37.5%)
Mutual labels:  question-answering, natural-language-processing

sentence-similarity

I plan to implement some models for sentence similarity found in the literature to reproduce and study them. They have a wide variety of application, including:

  • Paraphrase Detection: Give two sentences, are the sentences paraphrases of each other?
  • Semantic Texual Similarity: Given two sentences, how close are they in terms of semantic equivalence?
  • Natural Language Inference / Textual Entailment: Can one sentence be inferred from another sentence (the premise)?
  • Answer Selection: Given question-answer pairs, rank candidate answers based on relevance to question.

Setup

Install packages in requirements.txt.

Theignite library, currently in alpha, needs to be installed from source. See https://github.com/pytorch/ignite.

Download SpaCy English model:

python -m spacy download en

Compile trec_eval for computing MAP/MRR metrics for WikiQA dataset:

cd metrics
./get_trec_eval.sh

Running

Baseline

SICK

# Unsupervised
$ python main.py --model sif --dataset sick --unsupervised
Test Results - Epoch: 0 pearson: 0.7199 spearman: 0.5956
# Supervised
$ python main.py --model sif --dataset sick
Test Results - Epoch: 15 pearson: 0.7763 spearman: 0.6637
$ python main.py --model mpcnn --dataset sick
$ python main.py --model bimpm --dataset sick

WikiQA

$ python main.py --model sif --dataset wikiqa --epochs 15 --lr 0.001
Test Results - Epoch: 15 map: 0.6295 mrr: 0.6404
$ python main.py --model mpcnn --dataset wikiqa
$ python main.py --model bimpm --dataset wikiqa

Attribution

The English Wikipedia token frequency dataset for estimating p(w) in the baseline model is obtained from the official SIF implementation: https://github.com/PrincetonML/SIF.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].