Top 107 corpus open source projects

中文人名语料库。人名生成器。中文姓名,姓氏,名字,称呼,日本人名,翻译人名,英文人名。可用于中文分词、人名实体识别。

Deep Learning and deep reinforcement learning research papers and some codes

Final Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator

Cornell NLVR and NLVR2 are natural language grounding datasets. Each example shows a visual input and a sentence describing it, and is annotated with the truth-value of the sentence.

❤️Emotional First Aid Dataset, 心理咨询问答、聊天机器人语料库

A Curated List of Dataset and Usable Library Resources for NLP in Bahasa Indonesia

data resource untuk NLP bahasa indonesia

WP2TXT extracts plain text data from Wikipedia dump file (encoded in XML/compressed with Bzip2) stripping all the MediaWiki markups and other metadata.

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

Helsinki Prosody Corpus and A System for Predicting Prosodic Prominence from Text

PTT 八卦版問答中文語料

Preprocessed Python functions and docstrings for automated code documentation (code2doc) and automated code generation (doc2code) tasks.

Awesome Chatbot Projects,Corpus,Papers,Tutorials.Chinese Chatbot =>:

KH Coder: for Quantitative Content Analysis or Text Mining

搜索所有中文NLP数据集，附常用英文NLP数据集

用于训练中英文对话系统的语料库 Datasets for Training Chatbot System

A curated list of NLP resources for Hungarian

Korean sejong corpus download and simple analysis

Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate and query pattern models.

Poetry-related datasets developed by THUAIPoet (Jiuge) group.

UA-GEC: Grammatical Error Correction and Fluency Corpus for the Ukrainian Language

Tools for ASR Corpus Generation from Online Video

PubMed 200k RCT dataset: a large dataset for sequential sentence classification.

คลังศัพท์ภาษาไทย

迟先生语料库

Python toolkit for Chinese Language Understanding(CLUE) Evaluation benchmark

lists of text corpus and more (mainly Japanese)

Japanese text8 corpus for word embedding.

Russian mass media stemmed texts corpus / Корпус лемматизированных (морфологически нормализованных) текстов российских СМИ

A corpus retrieval engine based on Apache Lucene

Corpus of Annual Reports in Japan

Pre-trained Wikipedia corpus by MITIE

A multilingual dialog corpus

Typing Assistant provides the ability to autocomplete words and suggests predictions for the next word. This makes typing faster, more intelligent and reduces effort.

An unofficial Python API that allows users to create a corpus of lyrical text from their favorite artists and billboard charts

公司名语料库。机构名语料库。公司简称,缩写,品牌词,企业名。可用于中文分词、机构名实体识别。

Naive Bayes classifier is classification algorithm. It uses Naive based Bernoulli and Multinomial equation to classify documents(Text) as ham or spam.

🚁 保险行业语料库，聊天机器人

Chatbot in 200 lines of code using TensorLayer

大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP

An R package for the Quantitative Analysis of Textual Data

高质量中文预训练模型集合：最先进大模型、最快小模型、相似度专门模型

微信公众号语料库

Some useful Chinese corpus datasets 中文语料小数据

Curated List of Persian Natural Language Processing and Information Retrieval Tools and Resources

Crawl BookCorpus

Collections of Chinese NLP corpus

A collection of small corpuses of interesting data for the creation of bots and similar stuff.

An Integrated Corpus Tool With Multilingual Support for the Study of Language, Literature, and Translation

Fuzzing resources for feeding various fuzzers with input. 🔧

Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料

Korean corpus repository

A dataset of millions of news articles scraped from a curated list of data sources.

医疗语料库。医疗机构名语料库。药品本位码。

Curated list of publicly available parallel corpus for Indian Languages

Computer-generated poetry

extract relationships from standardized terms from corpus of interest with deep learning 🐟

Fast corpus search engine originally made for the Corpus of Written Tatar language

Open Discourse is the first fully comprehensive corpus of the plenary proceedings of the federal German Parliament (Bundestag).

物种名称语料库。植物名,动物名。