Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

🤖 Dev tools to reliably understand text and automate conversations. Built-in NLU. Connect & deploy on any messaging channel (Slack, MS Teams, website, Telegram, etc).

Stars: ✭ 9,486 (+6724.46%)

Mutual labels: chatbots

Chat

Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots

Stars: ✭ 8,238 (+5826.62%)

Mutual labels: chatbots

Rasa Chatbot Templates

RASA chatbot use case boilerplate

Stars: ✭ 127 (-8.63%)

Mutual labels: chatbots

Blindchat

a facebook messenger bot that allows users to chat with other people on facebook anonymously

Stars: ✭ 78 (-43.88%)

Mutual labels: chatbots

Xatkit

The simplest way to build all types of smart chatbots and digital assistants

Stars: ✭ 97 (-30.22%)

Mutual labels: chatbots

Bot Context

Easy, powerful, functional way to maintain conversational context in chat bots.

Stars: ✭ 44 (-68.35%)

Mutual labels: chatbots

Convai Bot 1337

NIPS Conversational Intelligence Challenge 2017 Winner System: Skill-based Conversational Agent with Supervised Dialog Manager

Stars: ✭ 65 (-53.24%)

Mutual labels: chatbots

Wa Automate Nodejs

💬 🤖 The most advanced NodeJS WhatsApp library for chatbots with advanced features. Be sure to 🌟 this repository for updates!

Stars: ✭ 1,326 (+853.96%)

Mutual labels: chatbots

Rasa stack quickstart

🚀 Easiest way to get started with Rasa stack. A full stack web based chatbot on top of rasa stack.

Stars: ✭ 16 (-88.49%)

Mutual labels: chatbots

Ai Chatbot Framework

A python chatbot framework with Natural Language Understanding and Artificial Intelligence.

Stars: ✭ 1,564 (+1025.18%)

Mutual labels: chatbots

Chatito

🎯🗯 Generate datasets for AI chatbots, NLP tasks, named entity recognition or text classification models using a simple DSL!

Stars: ✭ 678 (+387.77%)

Mutual labels: chatbots

Botbuilder Community Js

Part of the Bot Builder Community Project. Repository for extensions for the Bot Builder JavaScript SDK, including middleware, dialogs, recognizers and more.

Stars: ✭ 88 (-36.69%)

Mutual labels: chatbots

Rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

Stars: ✭ 13,219 (+9410.07%)

Mutual labels: chatbots

Intermediator Bot Sample

A sample bot, built with the Microsoft Bot Framework (v4), that routes messages between two users on different channels.

Stars: ✭ 120 (-13.67%)

Mutual labels: chatbots

Botfuel Dialog

Botfuel SDK to build highly conversational chatbots

Stars: ✭ 96 (-30.94%)

Mutual labels: chatbots

View All Similar Projects ➔

Multi-Turn Response Selection in Retrieval-Based Chatbots

Multi-turn response selection in retrieval-based chatbots is a task which aims to select the best-matched response from a set of candidates, given the context of a conversation. This task is attracting more and more attention in academia and industry. However, no one has maintained a leaderboard and a collection of popular papers and datasets yet. The main objective of this repository is to provide the reader with a quick overview of benchmark datasets and the state-of-the-art studies on this task, which serves as a stepping stone for further research.

Datasets

Ubuntu Dialogue Corpus V1

Lowe et al. (2015) released the original version. [Download]
Xu et al. (2016) released a version in which numbers, paths and URLs were replaced by placeholders. [Download]
Gu et al. (2019) released a version based on Xu et al. (2016) in which contexts and responses are assigned ids to track them. [Download]

Ubuntu Dialogue Corpus V2

Lowe et al. (2017) released the original version. [Download]
Gu et al. (2019) released a version in which contexts and responses are assigned ids to track them. [Download]

Douban Conversation Corpus

Wu et al. (2017) released the original version. [Download]
Gu et al. (2019) released a version in which contexts and responses are assigned ids to track them. [Download]

E-commerce Corpus

Zhang et al. (2018) released the original version. [Download]
Gu et al. (2019) released a version in which contexts and responses are assigned ids to track them. [Download]

Leaderboard

Ubuntu Dialogue Corpus V1

Model	[email protected]	[email protected]	[email protected]	[email protected]	Paper and Code
BERT-SL (Xu et al., 2020)	0.975	0.884	0.946	0.990	Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues. AAAI 2021.
UMS_BERT+ (Whang et al., 2020)	-	0.875	0.942	0.988	Do Response Selection Models Really Know What’s Next? Utterance Manipulation Strategies for Multi-turn Response Selection. AAAI 2021.
DCM (Li et al., 2020)	-	0.868	0.936	0.987	Deep context modeling for multi-turn response selection in dialogue systems. Information Processing & Management 2020.
SA-BERT (Gu et al., 2020)	0.965	0.855	0.928	0.983	Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2020.
BERT-VFT (Whang et al., 2019)	-	0.855	0.928	0.985	An Effective Domain Adaptive Post-Training Method for BERT in Response Selection. INTERSPEECH 2020.
RoBERTa-BASE-SS-DA (Lu et al., 2020)	0.955	0.826	0.909	0.978	Improving Contextual Language Models for Response Retrieval in Multi-Turn Conversation. SIGIR 2020.
TADAM (Xu et al., 2020)	-	0.821	0.906	0.978	Topic-Aware Multi-turn Dialogue Modeling. AAAI 2021.
G-MSN (Lin et al., 2020)	0.958	0.812	0.911	0.987	The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection. EMNLP 2020.
MSN (Yuan et al., 2019)	-	0.800	0.899	0.978	Multi-hop Selector Network for Multi-turn Response Selection in Retrieval-based Chatbots. EMNLP 2019.
IOI (Tao et al., 2019)	0.947	0.796	0.894	0.974	One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues. ACL 2019.
IMN (Gu et al., 2019)	0.946	0.794	0.889	0.974	Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2019.
U2U-IMN (Gu et al., 2019)	0.945	0.790	0.886	0.973	Utterance-to-Utterance Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. TASLP 2019.
MRFN (Tao et al., 2019)	0.945	0.786	0.886	0.976	Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. WSDM 2019.
IACMN (Wang et al., 2019)	0.944	0.782	0.886	0.973	Multi-Turn Response Selection in Retrieval-Based Chatbots with Iterated Attentive Convolution Matching Network. CIKM 2019
DAM (Zhou et al., 2018)	0.938	0.767	0.874	0.969	Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network. ACL 2018.
DUA (Zhang et al., 2018)	-	0.752	0.868	0.962	Modeling Multi-Turn Conversation with Deep Utterance Aggregation. COLING 2018.
SMN (Wu et al., 2017)	0.926	0.726	0.847	0.961	Sequential Matching Network: A New Architecture for Multi-Turn Response Selection in Retrieval-Based Chatbots. ACL 2017.

Ubuntu Dialogue Corpus V2

Model	[email protected]	[email protected]	[email protected]	[email protected]	Paper and Code
Cross-encoder (Humeau et al., 2020)	-	0.865	-	0.991	Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. ICLR 2020.
Thread-bi (Jia et al., 2020)	-	0.838	0.924	0.985	Multi-turn Response Selection using Dialogue Dependency Relations. EMNLP 2020.
SA-BERT (Gu et al., 2020)	0.963	0.830	0.919	0.985	Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2020.
IMN (Gu et al., 2019)	0.945	0.771	0.886	0.979	Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2019.
U2U-IMN (Gu et al., 2019)	0.943	0.762	0.877	0.975	Utterance-to-Utterance Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. TASLP 2019.
HRDE-LTC (Yoon et al., 2018)	0.915	0.652	0.815	0.966	Learning to Rank Question-Answer Pairs using Hierarchical Recurrent Encoder with Latent Topic Clustering. NAACL 2018.

Douban Conversation Corpus

Model	MAP	MRR	[email protected]	[email protected]	[email protected]	[email protected]	Paper and Code
UMS_BERT+ (Whang et al., 2020)	0.625	0.664	0.499	0.318	0.482	0.858	Do Response Selection Models Really Know What’s Next? Utterance Manipulation Strategies for Multi-turn Response Selection. AAAI 2021.
SA-BERT (Gu et al., 2020)	0.619	0.659	0.496	0.313	0.481	0.847	Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2020.
DCM (Li et al., 2020)	0.611	0.649	-	0.294	0.498	0.842	Deep context modeling for multi-turn response selection in dialogue systems. Information Processing & Management 2020.
RoBERTa-BASE-SS-DA (Lu et al., 2020)	0.602	0.646	0.460	0.280	0.495	0.847	Improving Contextual Language Models for Response Retrieval in Multi-Turn Conversation. SIGIR 2020.
G-MSN (Lin et al., 2020)	0.599	0.645	0.476	0.308	0.468	0.826	The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection. EMNLP 2020.
TADAM (Xu et al., 2020)	0.594	0.633	0.453	0.282	0.472	0.828	Topic-Aware Multi-turn Dialogue Modeling. AAAI 2021.
MSN (Yuan et al., 2019)	0.587	0.632	0.470	0.295	0.452	0.788	Multi-hop Selector Network for Multi-turn Response Selection in Retrieval-based Chatbots. EMNLP 2019.
IOI (Tao et al., 2019)	0.573	0.621	0.444	0.269	0.451	0.786	One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues. ACL 2019.
IACMN (Wang et al., 2019)	0.571	0.621	0.448	0.269	0.453	0.783	Multi-Turn Response Selection in Retrieval-Based Chatbots with Iterated Attentive Convolution Matching Network. CIKM 2019.
MRFN (Tao et al., 2019)	0.571	0.617	0.448	0.276	0.435	0.783	Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. WSDM 2019.
IMN (Gu et al., 2019)	0.570	0.615	0.433	0.262	0.452	0.789	Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2019.
U2U-IMN (Gu et al., 2019)	0.564	0.611	0.429	0.259	0.430	0.791	Utterance-to-Utterance Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. TASLP 2019.
DAM (Zhou et al., 2018)	0.550	0.601	0.427	0.254	0.410	0.757	Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network. ACL 2018.
DUA (Zhang et al., 2018)	0.551	0.599	0.421	0.243	0.421	0.780	Modeling Multi-Turn Conversation with Deep Utterance Aggregation. COLING 2018.
SMN (Wu et al., 2017)	0.529	0.569	0.397	0.233	0.396	0.724	Sequential Matching Network: A New Architecture for Multi-Turn Response Selection in Retrieval-Based Chatbots. ACL 2017.

E-commerce Corpus

Model	[email protected]	[email protected]	[email protected]	Paper and Code
BERT-SL (Xu et al., 2020)	0.776	0.919	0.991	Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues. AAAI 2021.
UMS_BERT+ (Whang et al., 2020)	0.762	0.905	0.986	Do Response Selection Models Really Know What’s Next? Utterance Manipulation Strategies for Multi-turn Response Selection. AAAI 2021.
SA-BERT (Gu et al., 2020)	0.704	0.879	0.985	Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2020.
DCM (Li et al., 2020)	0.685	0.864	0.982	Deep context modeling for multi-turn response selection in dialogue systems. Information Processing & Management 2020.
TADAM (Xu et al., 2020)	0.660	0.834	0.975	Topic-Aware Multi-turn Dialogue Modeling. AAAI 2021.
RoBERTa-BASE-SS-DA (Lu et al., 2020)	0.627	0.835	0.980	Improving Contextual Language Models for Response Retrieval in Multi-Turn Conversation. SIGIR 2020.
IMN (Gu et al., 2019)	0.621	0.797	0.964	Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. CIKM 2019.
U2U-IMN (Gu et al., 2019)	0.616	0.806	0.966	Utterance-to-Utterance Interactive Matching Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. TASLP 2019.
G-MSN (Lin et al., 2020)	0.613	0.786	0.964	The World is Not Binary: Learning to Rank with Grayscale Data for Dialogue Response Selection. EMNLP 2020.
MSN (Yuan et al., 2019)	0.606	0.770	0.937	Multi-hop Selector Network for Multi-turn Response Selection in Retrieval-based Chatbots. EMNLP 2019.
IOI (Tao et al., 2019)	0.563	0.768	0.950	One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues. ACL 2019.
DUA (Zhang et al., 2018)	0.501	0.700	0.921	Modeling Multi-Turn Conversation with Deep Utterance Aggregation. COLING 2018.
SMN (Wu et al., 2017)	0.453	0.654	0.886	Sequential Matching Network: A New Architecture for Multi-Turn Response Selection in Retrieval-Based Chatbots. ACL 2017.

Papers

In addition to the studies mentioned above, there are stil a lot of great studies on multi-turn response selection worth reading. We list a part of them below.

Distilling Knowledge for Fast Retrieval-based Chat-bots. Amir Vakili Tahami, Kamyar Ghajar, Azadeh Shakery. SIGIR 2020.
Improving Matching Models with Hierarchical Contextualized Representations for Multi-turn Response Selection. Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, Rui Yan. SIGIR 2020.
Conversational Word Embedding for Retrieval-Based Dialog System. Wentao Ma, Yiming Cui, Ting Liu, Dong Wang, ShijinWang, Guoping Hu. ACL 2020.
IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems. Liu Yang, Minghui Qiu, Chen Qu, Cen Chen, Jiafeng Guo, Yongfeng Zhang, W. Bruce Croft, Haiqing Chen. WWW 2020.
Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbots. Jia-Chen Gu, Zhen-Hua Ling, Xiaodan Zhu, Quan Liu. EMNLP 2019.
Sampling Matters! An Empirical Study of Negative Sampling Strategies for Learning of Matching Models in Retrieval-based Dialogue Systems. Jia Li, Chongyang Tao, wei wu, Yansong Feng, Dongyan Zhao, Rui Yan. EMNLP 2019.
Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems. Jiazhan Feng, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, Rui Yan. ACL 2019.
Training Neural Response Selection for Task-Oriented Dialogue Systems. Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, Pei-Hao Su. ACL 2019.
DSTC7 Task 1: Noetic End-to-End Response Selection. Chulaka Gunasekara, Jonathan K. Kummerfeld, Lazaros Polymenakos, Walter Lasecki. ACL 2019 Workshop.
A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots. Xueliang Zhao, Chongyang Tao, Wei Wu, Can Xu, Dongyan Zhao, Rui Yan. IJCAI 2019.
Sequential Attention-based Network for Noetic End-to-End Response Selection. Qian Chen, Wen Wang. AAAI 2019 Workshop on DSTC 7.
Building Sequential Inference Models for End-to-End Response Selection. Jia-Chen Gu, Zhen-Hua Ling, Yuping Ruan, Quan Liu. AAAI 2019 Workshop on DSTC 7.
Training Millions of Personalized Dialogue Agents. Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, Antoine Bordes. EMNLP 2018.
Personalizing Dialogue Agents: I have a dog, do you have pets too?. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, Jason Weston. ACL 2018.
Learning Matching Models with Weak Supervision for Response Selection in Retrieval-based Chatbots. Yu Wu, Wei Wu, Zhoujun Li, Ming Zhou. ACL 2018.
Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems. Liu Yang, Minghui Qiu, Chen Qu, Jiafeng Guo, Yongfeng Zhang, W. Bruce Croft, Jun Huang, Haiqing Chen. SIGIR 2018.
Improving Response Selection in Multi-turn Dialogue Systems by Incorporating Domain Knowledge. Debanjan Chaudhuri, Agustinus Kristiadi, Jens Lehmann, Asja Fischer. CONLL 2018.
Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus. Jianxiong Dong, Jim Huang. ArXiv.
Multi-view Response Selection for Human-Computer Conversation. Xiangyang Zhou, Daxiang Dong, Hua Wu, Shiqi Zhao, Dianhai Yu, Hao Tian, Xuan Liu, Rui Yan. EMNLP 2016.
Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System. Rui Yan, Yiping Song, Hua Wu. SIGIR 2016.

Update

Although we work very hard to list more work, the studies we select to present in this repository are by no means complete. To this end, we welcome more people to participate in the maintenance of this project. Please feel free to open issues, pull requests or contact us ([email protected]).

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].

Stars: ✭ 139

Visit Git Page 🔗Visit User Page 🔗Visit Issues Page (0) 🔗