1136 Open source projects that are alternatives of or similar to Tokenizers

Universal 1d/2d data containers with Transformers functionality for data analysis.

Vietnamese NLP Toolkit for Node

中文wiki百科QA阅读理解问答系统，使用了CCKS2016数据的NER模型和CMRC2018的阅读理解模型，还有W2V词向量搜索,使用torchserve部署

Collection of stats, modeling, and data science tools in Python and R.

An open source implementation of CLIP.

Data and software for building the ACL Anthology.

Simple downloader for pre-trained word vectors

⚡️A Lightweight NLP Data Loader for All Deep Learning Frameworks in Python

Useful resources for Mongolian NLP

🐍 Turkish Language Stemmer for Python

TEXTOIR is a flexible toolkit for open intent detection and discovery. (ACL 2021)

A news recommendation system tailored for user communities

Code Generation as a Dual Task of Code Summarization.

A text analytics library with support for codemixed data

Browse Covid-19 & SARS-CoV-2 Scientific Papers with Transformers 🦠 📖

EVE bot, a customer service chatbot to enhance virtual engagement for Twitter Apple Support

Fine-grained Sentiment Classification Using BERT

A Curated List of Dataset and Usable Library Resources for NLP in Bahasa Indonesia

text generation based on transformer

Basic Utilities for PyTorch Natural Language Processing (NLP)

COLING'2020: CoLAKE: Contextualized Language and Knowledge Embedding

Mishkal is an arabic text vocalization software

⏰ AI conference deadline countdowns

Topic Modelling for Humans

Chatbot with bert chinese model, base on rasa framework（中文聊天机器人，结合bert意图分析，基于rasa框架）

BERT, LDA, and TFIDF based keyword extraction in Python

PyTorch code for Learning Cooperative Visual Dialog Agents using Deep Reinforcement Learning

Curated List of Persian Natural Language Processing and Information Retrieval Tools and Resources

Complete solutions for Stanford CS224n, winter, 2019

End-to-end ASR/LM implementation with PyTorch

Neural question generation using transformers

Language Modeling and Text Classification in Malayalam Language using ULMFiT

Streaming based VHDL parser.

A Robustly Optimized BERT Pretraining Approach for Vietnamese

How to build RNNs and LSTMs from scratch with NumPy.

Alpine Linux Setup Scripts

基于深度学习的自然语言处理库

Naver movie review sentiment classification with KoBERT

Examine two sentences and determine whether they have the same meaning.

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

A Library to parse natural language in pure Clojure and ClojureScript

A Personal Arch Installation Guide In Case of Amnesia

Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE)

Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GanFormer and TransGan paper

💙 Emoji handling and meta data for spaCy with custom extension attributes

Chinese Natural Language Correction via Language Model

A CS degree with a focus on full-stack ML engineering, 2020

Cantonese Linguistics and NLP in Python

Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch

Implementation of the paper Tree Transformer

Transformer based Bangla Speech Recognition

RNN(LSTM, GRU) in Theano with mini-batch training; character-level language models in Theano

Deep Learning Transformer models in MATLAB

Library - Vanilla, ViT, DeiT, BERT, GPT

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

TensorFlow implementation of "Tracking the World State with Recurrent Entity Networks".

MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems

🏭 PDF text extraction pipeline: self-hosted, local-first, Docker-based

Implementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021

KoBERT on 🤗 Huggingface Transformers 🤗 (with Bug Fixed)