GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → nlpdata → dialogre

nlpdata / dialogre

Licence: other
Dialogue-Based Relation Extraction

Programming Languages

python
139335 projects - #7 most used programming language

Labels

dialoguedatasetrelation-extraction

Projects that are alternatives of or similar to dialogre

Tgen
Statistical NLG for spoken dialogue systems
Stars: ✭ 179 (+44.35%)
Mutual labels:  dialogue
OpenUE
OpenUE是一个轻量级知识图谱抽取工具 (An Open Toolkit for Universal Extraction from Text published at EMNLP2020: https://aclanthology.org/2020.emnlp-demos.1.pdf)
Stars: ✭ 274 (+120.97%)
Mutual labels:  relation-extraction
SENet-for-Weakly-Supervised-Relation-Extraction
No description or website provided.
Stars: ✭ 39 (-68.55%)
Mutual labels:  relation-extraction
Convlab 2
ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems
Stars: ✭ 196 (+58.06%)
Mutual labels:  dialogue
Dialogue-Corpus
No description or website provided.
Stars: ✭ 27 (-78.23%)
Mutual labels:  dialogue
Probabilistic-RNN-DA-Classifier
Probabilistic Dialogue Act Classification for the Switchboard Corpus using an LSTM model
Stars: ✭ 22 (-82.26%)
Mutual labels:  dialogue
Sequicity
Source code for the ACL 2018 paper entitled "Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures" by Wenqiang Lei et al.
Stars: ✭ 145 (+16.94%)
Mutual labels:  dialogue
rasa ch faq
用 rasa 实现 rasa demo 机器人，有一些惊奇的功能，faq，图谱，多轮等
Stars: ✭ 156 (+25.81%)
Mutual labels:  dialogue
GP-GNN
Code and dataset of ACL2019 Paper: Graph Neural Networks with Generated Parameters for Relation Extraction.
Stars: ✭ 52 (-58.06%)
Mutual labels:  relation-extraction
ReQuest
Indirect Supervision for Relation Extraction Using Question-Answer Pairs (WSDM'18)
Stars: ✭ 26 (-79.03%)
Mutual labels:  relation-extraction
Dialogue
Node based dialogue system
Stars: ✭ 207 (+66.94%)
Mutual labels:  dialogue
ConDigSum
Code for EMNLP 2021 paper "Topic-Aware Contrastive Learning for Abstractive Dialogue Summarization"
Stars: ✭ 62 (-50%)
Mutual labels:  dialogue
spert
PyTorch code for SpERT: Span-based Entity and Relation Transformer
Stars: ✭ 572 (+361.29%)
Mutual labels:  relation-extraction
Anyq
FAQ-based Question Answering System
Stars: ✭ 2,336 (+1783.87%)
Mutual labels:  dialogue
R-BERT
Pytorch re-implementation of R-BERT model
Stars: ✭ 59 (-52.42%)
Mutual labels:  relation-extraction
Deeputteranceaggregation
Modeling Multi-turn Conversation with Deep Utterance Aggregation (COLING 2018)
Stars: ✭ 171 (+37.9%)
Mutual labels:  dialogue
lima
The Libre Multilingual Analyzer, a Natural Language Processing (NLP) C++ toolkit.
Stars: ✭ 75 (-39.52%)
Mutual labels:  relation-extraction
DocuNet
Code and dataset for the IJCAI 2021 paper "Document-level Relation Extraction as Semantic Segmentation".
Stars: ✭ 84 (-32.26%)
Mutual labels:  relation-extraction
Chatbot-Training-Corpus
总结了一些可以用作聊天机器人训练实作的文字语聊，包含中英文不同语言
Stars: ✭ 117 (-5.65%)
Mutual labels:  dialogue
Att-BLSTM-relation-extraction
Implementation of Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification.
Stars: ✭ 60 (-51.61%)
Mutual labels:  relation-extraction
View All Similar Projects ➔

DialogRE

This repository maintains DialogRE, the first human-annotated dialogue-based relation extraction dataset. Please contact [email protected] if you have any questions or suggestions.

@inproceedings{yu2020dialogue,
  title={Dialogue-Based Relation Extraction},
  author={Yu, Dian and Sun, Kai and Cardie, Claire and Yu, Dong},
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year={2020},
  url={https://arxiv.org/abs/2004.08056v1}
}

Files in this repository:

  • license.txt: the license of DialogRE.
  • data/{train,dev,test}.json: the original dataset files (used in the paper). The data format is as follows.
[
  [
    [
      dialogue 1 / turn 1,
      dialogue 1 / turn 2,
      ...
    ],
    [
      {
        "x": dialogue 1 / instance 1 / argument 1,
        "y": dialogue 1 / instance 1 / argument 2,
        "x_type": dialogue 1 / instance 1 / argument 1 type,
        "y_type": dialogue 1 / instance 1 / argument 2 type,
        "r": [
          dialogue 1 / instance 1 / relation 1,
          dialogue 1 / instance 1 / relation 2,
          ...
        ],
        "rid": [
          dialogue 1 / instance 1 / relation 1 id,
          dialogue 1 / instance 1 / relation 2 id,
          ...
        ],
        "t": [
          dialogue 1 / instance 1 / relation 1 trigger,
          dialogue 1 / instance 1 / relation 2 trigger,
          ...
        ],
      },
      {
        "x": dialogue 1 / instance 2 / argument 1,
        "y": dialogue 1 / instance 2 / argument 2,
        "x_type": dialogue 1 / instance 2 / argument 1 type,
        "y_type": dialogue 1 / instance 2 / argument 2 type,
        "r": [
          dialogue 1 / instance 2 / relation 1,
          dialogue 1 / instance 2 / relation 2,
          ...
        ],
        "rid": [
          dialogue 1 / instance 2 / relation 1 id,
          dialogue 1 / instance 2 / relation 2 id,
          ...
        ],
        "t": [
          dialogue 1 / instance 2 / relation 1 trigger,
          dialogue 1 / instance 2 / relation 2 trigger,
          ...
        ],
      },
      ...
    ],
  ],
  [
    [
      dialogue 2 / turn 1,
      dialogue 2 / turn 2,
      ...
    ],
    [
      {
        "x": dialogue 2 / instance 1 / argument 1,
        "y": dialogue 2 / instance 1 / argument 2,
        "x_type": dialogue 2 / instance 1 / argument 1 type,
        "y_type": dialogue 2 / instance 1 / argument 2 type,
        "r": [
          dialogue 2 / instance 1 / relation 1,
          dialogue 2 / instance 1 / relation 2,
          ...
        ],
        "rid": [
          dialogue 2 / instance 1 / relation 1 id,
          dialogue 2 / instance 1 / relation 2 id,
          ...
        ],
        "t": [
          dialogue 2 / instance 1 / relation 1 trigger,
          dialogue 2 / instance 1 / relation 2 trigger,
          ...
        ],
      },
      {
        "x": dialogue 2 / instance 2 / argument 1,
        "y": dialogue 2 / instance 2 / argument 2,
        "x_type": dialogue 2 / instance 2 / argument 1 type,
        "y_type": dialogue 2 / instance 2 / argument 2 type,
        "r": [
          dialogue 2 / instance 2 / relation 1,
          dialogue 2 / instance 2 / relation 2,
          ...
        ],
        "rid": [
          dialogue 2 / instance 2 / relation 1 id,
          dialogue 2 / instance 2 / relation 2 id,
          ...
        ],
        "t": [
          dialogue 2 / instance 2 / relation 1 trigger,
          dialogue 2 / instance 2 / relation 2 trigger,
          ...
        ],
      },
      ...
    ],
  ],
  ...
]
  • data_v2/en/data/{train,dev,test}.json: the updated dataset files with a few annotation errors fixed. The format is the same as the orignal. (Updated on Aug 2020)
  • data_v2/cn/data/{train,dev,test}.json: a Chinese version of DialogRE. The format is the same as the orignal. Please note that since ground truth argument types do not substantially contribute to the performance according to Section 5.2 of our paper, we no longer annotate argument types when annotating the Chinese version. Instead, all "x_type"s and "y_type"s are left empty. (Updated on Aug 2020)
  • kb/Fandom_triples: relational triples from Fandom.
  • kb/matching_table.txt: mapping from Fandom relational types to DialogRE relation types.
  • bert folder: a re-implementation of BERT and BERTS baselines.
    1. Download and unzip BERT from here, and set up the environment variable for BERT by export BERT_BASE_DIR=/PATH/TO/BERT/DIR.
    2. Copy the dataset folder data (or data_v2/{en,cn}/data for the updated version) to bert/.
    3. In bert, execute python convert_tf_checkpoint_to_pytorch.py --tf_checkpoint_path=$BERT_BASE_DIR/bert_model.ckpt --bert_config_file=$BERT_BASE_DIR/bert_config.json --pytorch_dump_path=$BERT_BASE_DIR/pytorch_model.bin.
    4. To run and evaluate the BERT baseline, execute the following commands in bert:
    python run_classifier.py   --task_name bert  --do_train --do_eval   --data_dir .   --vocab_file $BERT_BASE_DIR/vocab.txt   --bert_config_file $BERT_BASE_DIR/bert_config.json   --init_checkpoint $BERT_BASE_DIR/pytorch_model.bin   --max_seq_length 512   --train_batch_size 24   --learning_rate 3e-5   --num_train_epochs 20.0   --output_dir bert_f1  --gradient_accumulation_steps 2
rm bert_f1/model_best.pt && cp -r bert_f1 bert_f1c && python run_classifier.py   --task_name bertf1c --do_eval   --data_dir .   --vocab_file $BERT_BASE_DIR/vocab.txt   --bert_config_file $BERT_BASE_DIR/bert_config.json   --init_checkpoint $BERT_BASE_DIR/pytorch_model.bin   --max_seq_length 512   --train_batch_size 24   --learning_rate 3e-5   --num_train_epochs 20.0   --output_dir bert_f1c  --gradient_accumulation_steps 2
python evaluate.py --f1dev bert_f1/logits_dev.txt --f1test bert_f1/logits_test.txt --f1cdev bert_f1c/logits_dev.txt --f1ctest bert_f1c/logits_test.txt
    1. To run and evaluate the BERTS baseline, execute the following commands in bert:
    python run_classifier.py   --task_name berts  --do_train --do_eval   --data_dir .   --vocab_file $BERT_BASE_DIR/vocab.txt   --bert_config_file $BERT_BASE_DIR/bert_config.json   --init_checkpoint $BERT_BASE_DIR/pytorch_model.bin   --max_seq_length 512   --train_batch_size 24   --learning_rate 3e-5   --num_train_epochs 20.0   --output_dir berts_f1  --gradient_accumulation_steps 2
rm berts_f1/model_best.pt && cp -r berts_f1 berts_f1c && python run_classifier.py   --task_name bertsf1c --do_eval   --data_dir .   --vocab_file $BERT_BASE_DIR/vocab.txt   --bert_config_file $BERT_BASE_DIR/bert_config.json   --init_checkpoint $BERT_BASE_DIR/pytorch_model.bin   --max_seq_length 512   --train_batch_size 24   --learning_rate 3e-5   --num_train_epochs 20.0   --output_dir berts_f1c  --gradient_accumulation_steps 2
python evaluate.py --f1dev berts_f1/logits_dev.txt --f1test berts_f1/logits_test.txt --f1cdev berts_f1c/logits_dev.txt --f1ctest berts_f1c/logits_test.txt

Environment: The code has been tested with Python 3.6 and PyTorch 1.0.

TODO:

  • Release DialogRE
  • Release a Chinese version of DialogRE (summer 2020)
  • Fix the annotation errors in DialogRE and release an updated English version (summer 2020)
  • Baseline results for the updated version (here)
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].