1057 Open source projects that are alternatives of or similar to Tokenizer

Geotext extracts country and city mentions from text

Materials for deep NLP course

General Assembly's 2015 Data Science course in Washington, DC

Open Source REST API for named entity extraction, named entity linking, named entity disambiguation, recommendation & reconciliation of entities like persons, organizations and places for (semi)automatic semantic tagging & analysis of documents by linked data knowledge graph like SKOS thesaurus, RDF ontology, database(s) or list(s) of names

Text tokenization and sentence segmentation (segtok v2)

Mapping a variable-length sentence to a fixed-length vector using BERT model

Code for Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension

RASA chatbot use case boilerplate

NiuTrans.SMT is an open-source statistical machine translation system developed by a joint team from NLP Lab. at Northeastern University and the NiuTrans Team. The NiuTrans system is fully developed in C++ language. So it runs fast and uses less memory. Currently it supports phrase-based, hierarchical phrase-based and syntax-based (string-to-tree, tree-to-string and tree-to-tree) models for research-oriented studies.

Papers and Book to look at when starting NLP 📚

Pytorch implementation of "Graph Convolutional Networks for Text Classification"

A collection of Natural language processing pre-trained models.

Implementation of the multi feed-forward network architecture by Parikh et al. (2016) for Natural Language Inference.

A comprehensive reading list for Emotion Recognition in Conversations

Mining individual characters in multiparty dialogue

Preprocessing Library for Natural Language Processing

Models and Pipelines for the Spark NLP library

XLNet Extension in TensorFlow

Cornell Semantic Parsing Framework

Code examples in pyTorch and Tensorflow for CS230

A high-level machine learning and deep learning library for the PHP language.

A comprehensive reference for all topics related to Natural Language Processing

Simple yet helpful library for writing Django urls by an easy, short and intuitive way.

PyTorch implementation of latent space reinforcement learning for E2E dialog published at NAACL 2019. It is released by Tiancheng Zhao (Tony) from Dialog Research Center, LTI, CMU

Easy (and fast) unicode string rendering addon for OpenFrameworks. FontStash is made by Andreas Krinke and Mikko Mononen

My personal notes and surveys on DL, CV and NLP papers.

[ECCV 2020] ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language

RAT-el is an open source penetration test tool that allows you to take control of a windows machine. It works on the client-server model, the server sends commands and the client executes the commands and sends the result back to the server. The client is completely undetectable by anti-virus software.

Unicode To Chinese -- U2C : A burpsuite Extender That Convert Unicode To Chinese 【Unicode编码转中文的burp插件】

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

Persian (Farsi) Stop Words List

Unicode normalization library. (Mirror of Yoshida-san's code base to maintain the RubyGem.)

NLTK Source

Visualize data using relative ordering

NLPCC 2016 微博分词评测项目

MIMIC Code Repository: Code shared by the research community for the MIMIC-III database

Русскоязычный чатбот

Deep Learning model to analyze a large corpus of clear text passwords.

Code for AAAI2020 paper "Graph Transformer for Graph-to-Sequence Learning"

💫 Scripts, tools and resources for developing spaCy

CogComp's light-weight Python NLP annotators

🧬 A JupyterLab extension for annotating data with Prodigy

搜索所有中文NLP数据集，附常用英文NLP数据集

Transformers without Tears: Improving the Normalization of Self-Attention

List of Machine Learning, AI, NLP solutions for iOS. The most recent version of this article can be found on my blog.

Unicode Database and Normalization

Code for paper "Discourse-Aware Neural Extractive Text Summarization" (ACL20)

🏄 依存关系分析，NLP，自然语言处理

Fine-tuned pre-trained GPT2 for custom topic specific text generation. Such system can be used for Text Augmentation.

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

NLP, before and after spaCy

A collection of 500+ survey papers on Natural Language Processing (NLP) and Machine Learning (ML)

A fast, efficient universal vector embedding utility package.

Official Ruby client for the MonkeyLearn API. Build and consume machine learning models for language processing from your Ruby apps.

Source code tokenizer

BOND: BERT-Assisted Open-Domain Name Entity Recognition with Distant Supervision

Proof of concept, general purpose pastejacker for GNU/Linux

Botfuel SDK to build highly conversational chatbots

PyTorch implementations of various deep learning models for paraphrase detection, semantic similarity, and textual entailment