1136 Open source projects that are alternatives of or similar to Tokenizers

This Word Does Not Exist

Cornell Instruction Following Framework

API for the GPT-J language model 🦜. Including a FastAPI backend and a streamlit frontend

Detect the sentiment captured in short pieces of text

The neural network model is capable of detecting five different male/female emotions from audio speeches. (Deep Learning, NLP, Python)

Data-centric declarative deep learning framework

Blocks World -- Simulator, Code, and Models (Misra et al. EMNLP 2017)

Electra pre-trained model using Vietnamese corpus

Programming assignments from all courses in the Coursera Natural Language Processing Specialization offered by deeplearning.ai.

Multi-Task Deep Neural Networks for Natural Language Understanding

Official Repository for 'Practical Natural Language Processing' by O'Reilly Media

A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning.

使用 RASA NLU 来构建中文自然语言理解系统（NLU）| Use RASA NLU to build a Chinese Natural Language Understanding System (NLU)

Rank 1 / 216

Anonymization of legal cases (Fr) based on Flair embeddings

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

A curated list of Open Information Extraction (OIE) resources: papers, code, data, etc.

Models and Pipelines for the Spark NLP library

Materials for deep NLP course

XLNet Extension in TensorFlow

A Greek edition of BERT pre-trained language model

Ruby SDK for Dialogflow

Pre-Trained Models for ToD-BERT

Library to scrape and clean web pages to create massive datasets.

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

a sklearn wrapper for Google's BERT model

基于自然语言理解与机器学习的聊天机器人，支持多用户并发及自定义多轮对话

Contextualised Embeddings and Language Modelling using BERT and Friends using R

[ACM-WSDM] 3rd place solution at WSDM Cup 2019, Fake News Classification on Kaggle.

A PyTorch-based toolkit for natural language processing

基于bert4keras的GLUE基准代码

[ACL-WS] 4th place solution to gendered pronoun resolution challenge on Kaggle

Tool for visualizing attention in the Transformer model (BERT, GPT-2, Albert, XLNet, RoBERTa, CTRL, etc.)

A repository to run gpt-j-6b on low vram machines (4.2 gb minimum vram for 2000 token context, 3.5 gb for 1000 token context). Model loading takes 12gb free ram.

Code & Data for "Tabular Transformers for Modeling Multivariate Time Series" (ICASSP, 2021)

Named Entity Recognition

BlueBERT, pre-trained on PubMed abstracts and clinical notes (MIMIC-III).

This repo is the home of Romanian Transformers.

This repository contains demos I made with the Transformers library by HuggingFace.

Transformer related optimization, including BERT, GPT

Utility for analyzing Transformer based representations of language.

Text2Text: Cross-lingual natural language processing and generation toolkit

🤗 AutoNLP: train state-of-the-art natural language processing models and deploy them in a scalable environment automatically

Abstractive and Extractive Text summarization using Transformers.

🚀 Catalyst is a C# Natural Language Processing library built for speed. Inspired by spaCy's design, it brings pre-trained models, out-of-the box support for training word and document embeddings, and flexible entity recognition models.

Persian Bert For Long-Range Sequences

Pytorch Implementation of GoEmotions 😍😢😱

Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"

Pytorch-NLU，一个中文文本分类、序列标注工具包，支持中文长文本、短文本的多类、多标签分类任务，支持中文命名实体识别、词性标注、分词等序列标注任务。 Ptorch NLU, a Chinese text classification and sequence annotation toolkit, supports multi class and multi label classification tasks of Chinese long text and short text, and supports sequence annotation tasks such as Chinese named entity recognition, part of speech ta…

Text Classification in Turkish Texts with Bert

Emotion recognition in conversation

Official Code for "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"

Bangla-Bert is a pretrained bert model for Bengali language

HugsVision is a easy to use huggingface wrapper for state-of-the-art computer vision

Implementation of BERT that could load official pre-trained models for feature extraction and prediction

A simplified PyTorch implementation of "SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient." (Yu, Lantao, et al.)

The source code of "Language Models are Few-shot Multilingual Learners" (MRL @ EMNLP 2021)

Build a Movie Reviews Sentiment Classifier with Google's BERT Language Model

An Open-Source Package for Chinese Open-domain Conversational Chatbot (中文闲聊对话系统，一键部署微信闲聊机器人)