159 Open source projects that are alternatives of or similar to Lex

C++ Lexer Toolkit Library (LexerTk) https://www.partow.net/programming/lexertk/index.html

Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.

Lex machinary for go.

Collection of developer toolkits

🌿 NodeJS PHP Parser - extract AST or tokens (PHP5 and PHP7)

SNL(Small Nested Language) Compiler. Maven jUnit Tokenizer Lexer Syntax Parser. 编译原理 词法分析 语法分析

Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.

A modern CSS value compiler in JavaScript

Parser Building Toolkit for JavaScript

The fast scanner generator for Java™ with full Unicode support

A universal lexer library in Swift.

Lex is an implementation of lex tool in Ruby.

A simple interpreter for a large subset of Pascal language written for educational purposes

A small .NET math library.

Simple yet helpful library for writing Django urls by an easy, short and intuitive way.

Naive Bayes text classification implementation as an OmniCat classifier strategy. (#ruby #naivebayes)

Laravel token management

Parse YARA rules and operate over them more easily.

Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

💠The Lel programming language

Mico ("Monkey" in catalan). Monkey language implementation done with C++. https://interpreterbook.com/

Colin's ALM Corner Custom Build Tasks

🌭 Mustard is a Swift library for tokenizing strings when splitting by whitespace doesn't cut it.

Self-contained Japanese Morphological Analyzer written in pure Go

DFA regular expression library & friends

C/C++ develop tool for android.

Fibers, Closures, C-Module System | NaN-boxing, bytecode-VM written in C++

A small library for converting tokenized PHP source code into XML (and potentially other formats)

NLP framework: sentence detector, tokeniser, pos-tagger and dependency parser

A tokenizer and sentence splitter for German and English web and social media texts.

POS Tagger, lemmatizer and stemmer for french language in javascript

Source code tokenizer

Fuzzing Parsers with Tokens

PHP standards checker.

React component for tagging inputs.

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

Monkey Programming Language written in Rust.

Solves basic Russian NLP tasks, API for lower level Natasha projects

Kadot, the unsupervised natural language processing library.

한국어 자연어처리를 위한 파이썬 라이브러리입니다. 단어 추출/ 토크나이저 / 품사판별/ 전처리의 기능을 제공합니다.

Ruby Object Inspection for IRB

Open Korean Text Processor - An Open-source Korean Text Processor

minigo🐥is a small Go compiler made from scratch. It can compile itself.

High performance tokenizers for natural language processing and other related tasks

a node-able bidirectionnal expression editor.

专注于可解释的NLP技术 An NLP Toolset With A Focus on Explainable Inference

A tool set for CSS including fast detailed parser, walker, generator and lexer based on W3C specs and browser implementations

Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).

PHP calculator library for mathematical terms (expressions) passed as strings

A tiny compiler for a language featuring LL(2) with Lexer, Parser, ASM-like codegen and VM. Complex enough to give you a flavour of how the "real" thing works whilst not being a mere toy example

Text tokenization and sentence segmentation (segtok v2)

😻 Markdown with easy tokenization, a fast highlighter, and a lean HTML sanitizer

Thot toolkit for statistical machine translation

Verible is a suite of SystemVerilog developer tools, including a parser, style-linter, and formatter.

The greynir.is natural language processing website for Icelandic

Light and fast 🚀parser! With zero dependents. - Sql Parser Demo added!

Fast lexer code generator for Rust

A collection of basic python modules for spoken natural language processing

High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other programs, like: MySQL, PostgreSQL, PHP, etc.

C++ Mathematical Expression Parsing And Evaluation Library