314 Open source projects that are alternatives of or similar to lexertk

Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.

A modern CSS value compiler in JavaScript

SNL(Small Nested Language) Compiler. Maven jUnit Tokenizer Lexer Syntax Parser. 编译原理 词法分析 语法分析

Replaced by foonathan/lexy

Collection of developer toolkits

React component for tagging inputs.

Lex is an implementation of lex tool in Ruby.

Lex machinary for go.

A universal lexer library in Swift.

Parser Building Toolkit for JavaScript

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.

🌿 NodeJS PHP Parser - extract AST or tokens (PHP5 and PHP7)

snapdragon is an extremely pluggable, powerful and easy-to-use parser-renderer factory.

A simple interpreter for a large subset of Pascal language written for educational purposes

The fast scanner generator for Java™ with full Unicode support

PHP standards checker.

Tiny JavaScript tokenizer.

Colin's ALM Corner Custom Build Tasks

PHP calculator library for mathematical terms (expressions) passed as strings

A collection of NLP tools for Sinhalese (සිංහල).

Fast, Consistent Tokenization of Natural Language Text

The greynir.is natural language processing website for Icelandic

NLP framework: sentence detector, tokeniser, pos-tagger and dependency parser

A small .NET math library.

POS Tagger, lemmatizer and stemmer for french language in javascript

Fuzzing Parsers with Tokens

Simple yet helpful library for writing Django urls by an easy, short and intuitive way.

Bitextor generates translation memories from multilingual websites.

Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

T-REX is a suite of smart contracts implementing the EIP 3643 and developed by Tokeny to manage and transfer financial assets on the Ethereum blockchain

Ruby Object Inspection for IRB

Query Translator is a search query translator with AST representation

Thot toolkit for statistical machine translation

Verify the Apple id token on the server side.

A collection of basic python modules for spoken natural language processing

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit

Command line utility for interacting with OAuth2 infrastructure to generate tokens

Essential NLP & ML, short & fast pure Python code

KAI is a distributed computing model written in modern C++ and is cross-plaftorm. Using custom language translators and an executor, KAI provides full reflection, persistence and cross-process communications without having to modify existing source code. KAI Comes with an automated, generational tricolor garbage collector, and Console- and Windo…

Microservice generates the pair of JSON web tokens - access-token and refresh-token are signed by user identifier.

Laravel token management

Naive Bayes text classification implementation as an OmniCat classifier strategy. (#ruby #naivebayes)

Fast and customizable text tokenization library with BPE and SentencePiece support

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

💠The Lel programming language

Go scripts for finding sensitive data like API key / some keywords in the github repository

Solves basic Russian NLP tasks, API for lower level Natasha projects

🌭 Mustard is a Swift library for tokenizing strings when splitting by whitespace doesn't cut it.

한국어 자연어처리를 위한 파이썬 라이브러리입니다. 단어 추출/ 토크나이저 / 품사판별/ 전처리의 기능을 제공합니다.

Text tokenization and sentence segmentation (segtok v2)

Self-contained Japanese Morphological Analyzer written in pure Go

A small library for converting tokenized PHP source code into XML (and potentially other formats)

Vietnamese tokenizer (Maximum Matching and CRF)

A Lexer and Parser for Valves Data Format (known as vdf) written in Go

aim to use JapaneseTokenizer as easy as possible

Open Korean Text Processor - An Open-source Korean Text Processor

专注于可解释的NLP技术 An NLP Toolset With A Focus on Explainable Inference

Source code tokenizer

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

Native Go bindings for OpenWrt's UCI.