1057 Open source projects that are alternatives of or similar to Tokenizer

[ECCV 2020] ScanRefer: 3D Object Localization in RGB-D Scans using Natural Language

A List of Data Science/Machine Learning Resources (Mostly Free)

RAT-el is an open source penetration test tool that allows you to take control of a windows machine. It works on the client-server model, the server sends commands and the client executes the commands and sends the result back to the server. The client is completely undetectable by anti-virus software.

Conversational AI Reading Materials

Unicode To Chinese -- U2C : A burpsuite Extender That Convert Unicode To Chinese 【Unicode编码转中文的burp插件】

MetaSRA: normalized sample-specific metadata for the Sequence Read Archive

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Paraphrase Generation model using pair-wise discriminator loss

Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

A large annotated semantic parsing corpus for developing natural language interfaces.

Persian (Farsi) Stop Words List

🔮Deep Learning for text made easy with AllenNLP

Unicode normalization library. (Mirror of Yoshida-san's code base to maintain the RubyGem.)

hexdump(1) for Unicode data

NLTK Source

An implementation of the NAACL 2018 paper "Punny Captions: Witty Wordplay in Image Descriptions".

Visualize data using relative ordering

Naive Bayes text classification implementation as an OmniCat classifier strategy. (#ruby #naivebayes)

NLPCC 2016 微博分词评测项目

DeepLearning.AI TensorFlow Developer Professional Certificate Specialization

MIMIC Code Repository: Code shared by the research community for the MIMIC-III database

Harfbuzz with a CMake build configuration using Freetype2, UCDN and ICU

Русскоязычный чатбот

Enables ICU message formatting for translations in Symfony applications.

Surgically polyfills timezone support in Intl.DateTimeFormat API

Record some papers I have read and paper notes I have taken, also including some awesome papers reading lists and academic blog posts.

A PHP port of https://github.com/vhf/confusable_homoglyphs

搜索所有中文NLP数据集，附常用英文NLP数据集

REx: Relation Extraction. Modernized re-write of the code in the master's thesis: "Relation Extraction using Distant Supervision, SVMs, and Probabalistic First-Order Logic"

Transformers without Tears: Improving the Normalization of Self-Attention

Witwicky: An implementation of Transformer in PyTorch.

List of Machine Learning, AI, NLP solutions for iOS. The most recent version of this article can be found on my blog.

name entity recognition with recurrent neural network(RNN) in tensorflow

Unicode Database and Normalization

Repository for *SEM Paper on Event Coreference Resolution in ECB+

Code for paper "Discourse-Aware Neural Extractive Text Summarization" (ACL20)

JavaScript-compatible Unicode data. Arrays of code points, arrays of symbols, and regular expressions for Unicode v9.0.0’s categories, scripts, blocks, bidi, and other properties.

🏄 依存关系分析，NLP，自然语言处理

📚 single header utf8 string functions for C and C++

Fine-tuned pre-trained GPT2 for custom topic specific text generation. Such system can be used for Text Augmentation.

jTransliter - the roman to unicode transliter as Google chrome extension

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data

[DEPRECATED] Ultimate Node.JS SDK for api.ai

NLP, before and after spaCy

Shared repository for open-sourced projects from the Google AI Language team.

A collection of 500+ survey papers on Natural Language Processing (NLP) and Machine Learning (ML)

Python Keyphrase Extraction module

A fast, efficient universal vector embedding utility package.

Self-contained Machine Learning and Natural Language Processing library in Go

Official Ruby client for the MonkeyLearn API. Build and consume machine learning models for language processing from your Ruby apps.

An algorithm to augment syntactic hierarchy into word embeddings

Source code tokenizer

Some JavaScript works published as demos, mostly ML or DS

NLP and Text Generation Experiments in TensorFlow 2.x / 1.x

😡😇 Stanford Sentiment Treebank loader in Python

Corpus of Annual Reports in Japan

Electra pre-trained model using Vietnamese corpus

Multitask Deep Learning for Sentiment Analysis using Character-Level Language Model, Bi-LSTMs for POS Tag, Chunking and Unsupervised Dependency Parsing. Inspired by this great article https://arxiv.org/abs/1611.01587

Python library for Myanmar text processing

A python code to detect emotions from text