437 Open source projects that are alternatives of or similar to chinese-tokenizer

邮箱 📧 newsletter RSS 荟萃 News

💠The Lel programming language

中文无监督SimCSE Pytorch实现

Solves basic Russian NLP tasks, API for lower level Natasha projects

中文预训练模型生成字向量学习，测试BERT，ELMO的中文效果

한국어 자연어처리를 위한 파이썬 라이브러리입니다. 단어 추출/ 토크나이저 / 품사판별/ 전처리의 기능을 제공합니다.

Bitextor generates translation memories from multilingual websites.

A small library for converting tokenized PHP source code into XML (and potentially other formats)

Essential NLP & ML, short & fast pure Python code

专注于可解释的NLP技术 An NLP Toolset With A Focus on Explainable Inference

Query Translator is a search query translator with AST representation

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.

A highly configurable vocabulary trainer

The fast scanner generator for Java™ with full Unicode support

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit

High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other programs, like: MySQL, PostgreSQL, PHP, etc.

網頁聊天機器人 | tensorflow implementation of seq2seq model with bahdanau attention and Word2Vec pretrained embedding

Python port of Moses tokenizer, truecaser and normalizer

Fast and customizable text tokenization library with BPE and SentencePiece support

A simple interpreter for a large subset of Pascal language written for educational purposes

A desktop web application for learning Mandarin Chinese and its character stroke order.

A very simple python tokenizer for Hebrew text.

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

A small .NET math library.

記錄有關繁化姬的議題或是內容

A modern CSS value compiler in JavaScript

Text tokenization and sentence segmentation (segtok v2)

Morphological analyzer `mystem` (Russian language) wrapper for JVM languages

Dialect identification using Siamese network

Tooling to play around with multilingual machine translation for Indian Languages.

Source code tokenizer

C library for Lexical Analysis

máobĭ (毛笔) is an Anki add-on to create cards with writing quizzes for Hanzi (Chinese characters)

Berserker - BERt chineSE woRd toKenizER

😻 Markdown with easy tokenization, a fast highlighter, and a lean HTML sanitizer

Some native scoring script plugins for elasticsearch

Chinese NLP Data Augmentation， BERT Contextual Augmentation

A Python implementation of Farasa toolkit

Simple yet helpful library for writing Django urls by an easy, short and intuitive way.

A PSR-2-R code sniffer and code-style auto-correction-tool - including many useful additions

Vietnamese tokenizer (Maximum Matching and CRF)

High-Performance Stemmer, Tokenizer, and Spell Checker for R

Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

A morphological analysis library.

Suika 🍉 is a Japanese morphological analyzer written in pure Ruby

A small godot project with a tokenizer written in GDScript.

Ruby Object Inspection for IRB

Get identifiers, paths, URLs and words from the previous command output and use them for the next command in xonsh shell.

A collection of NLP tools for Sinhalese (සිංහල).

Procedurally generated Chinese landscape painting.

Thot toolkit for statistical machine translation

Chinese text normalization for speech processing

A tokenizer for Icelandic text

A collection of basic python modules for spoken natural language processing

🏖️ Vanhiupun's Awesome Site ==> another theme for elegant writers with modern flat style and beautiful night/dark mode.

A Vim input method engine

Bag of words meets bags of popcorn in Python 3 中文教程

繁化姬的 Sublime Text 插件

Chinese font displaying Hanzi (汉字) characters with by transliteration/pronunciation (Pīnyīn).

POS Tagger, lemmatizer and stemmer for french language in javascript