132 Open source projects that are alternatives of or similar to cang-jie

High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other programs, like: MySQL, PostgreSQL, PHP, etc.

Rust re-implementation of OpenFST - library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs). A Python binding is also available.

Tokenizes Chinese texts into words.

The library web application where you can borrow books. It's Spring MVC and Hibernate project.

A repository to bind mecab for Python 3.5+. Not using swig nor pybind. (Not Maintained Now)

Some native scoring script plugins for elasticsearch

A tokenizer for Icelandic text

Tokenize CSS according to the CSS Syntax

Lex is an implementation of lex tool in Ruby.

Greeb is a simple Unicode-aware regexp-based tokenizer.

Fast, Consistent Tokenization of Natural Language Text

Full-text search with pre-build indexes for Nuxt.js using lunr.js

Berserker - BERt chineSE woRd toKenizER

Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.

Python Groonga Client

Suika 🍉 is a Japanese morphological analyzer written in pure Ruby

A Python implementation of Farasa toolkit

C++ Lexer Toolkit Library (LexerTk) https://www.partow.net/programming/lexertk/index.html

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

A collection of NLP tools for Sinhalese (සිංහල).

A PSR-2-R code sniffer and code-style auto-correction-tool - including many useful additions

Tiny JavaScript tokenizer.

VSCode extension to highlight nested code blocks

High-Performance Stemmer, Tokenizer, and Spell Checker for R

Replaced by foonathan/lexy

Collection of developer toolkits

A universal lexer library in Swift.

Multilingual tokenizer that automatically tags each token with its type

A small godot project with a tokenizer written in GDScript.

Gatsby plugin for full text search implementation based on lunr client-side index. Supports multilanguage search.

Get identifiers, paths, URLs and words from the previous command output and use them for the next command in xonsh shell.

Tokenizers and lemmatizers for Go

The code of "Making queries" in docs.djangoproject.com that I used in my article "Full-Text Search in Django with PostgreSQL".

A modern CSS value compiler in JavaScript

📖 Support examples for learning full-text search with use of PostgreSQL. Ready to run.

Tokenize English sentences using neural networks.

The aim of this repository is to show a baseline model for text classification by implementing a LSTM-based model coded in PyTorch. In order to provide a better understanding of the model, it will be used a Tweets dataset provided by Kaggle.

A supercharged version of paperless: scan, index and archive all your physical documents

Lightweight and read-write optimized full text search library.

Postgres full-text search in Node.js and Sequelize.

Essential NLP & ML, short & fast pure Python code

Text2Text: Cross-lingual natural language processing and generation toolkit

Vietnamese tokenizer (Maximum Matching and CRF)

🔍 基于讯搜封装的 Laravel 全文检索服务。

⚡ Insanely fast, 🌟 Feature-rich searching. lnx is the adaptable, typo tollerant deployment of the tantivy search engine. Standing on the shoulders of giants.

Tooling to play around with multilingual machine translation for Indian Languages.

Bitextor generates translation memories from multilingual websites.

A Code Index Searching Tools Based On Lucene.Net

Query Translator is a search query translator with AST representation

Morphological analyzer `mystem` (Russian language) wrapper for JVM languages

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit

🔍 Postgres full-text search (fts)

Fast and customizable text tokenization library with BPE and SentencePiece support

Fast Full Text Search based on BM25

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

A simple tokenizer in Ruby for NLP tasks.

使用 PaddlePaddle 实现基于深度神经网络的中文分词引擎 | A DNN Chinese Tokenizer by Using PaddlePaddle

Fast, efficient, in-memory Full Text Search for Kotlin

full text search manpages

C library for Lexical Analysis