196 Open source projects that are alternatives of or similar to ilmulti

Curated list of publicly available parallel corpus for Indian Languages

Thot toolkit for statistical machine translation

Python port of Moses tokenizer, truecaser and normalizer

Fast and customizable text tokenization library with BPE and SentencePiece support

A Python implementation of Farasa toolkit

Reader Translator Generator - NMT toolkit based on pytorch

Sanskrit compound segmentation using seq2seq model

Converts a string into an array of tokens, with useful methods for looking ahead and behind, capturing, matching, et cetera.

Berserker - BERt chineSE woRd toKenizER

A PSR-2-R code sniffer and code-style auto-correction-tool - including many useful additions

Tokenizes Chinese texts into words.

A universal lexer library in Swift.

Neural machine translation implementation using dynet's python bindings

A small godot project with a tokenizer written in GDScript.

A Fast Neural Machine Translation System. It is developed in C++ and resorts to NiuTensor for fast tensor APIs.

臺灣言語工具

PacketEvents is a powerful packet library. Our packet wrappers are efficient and easy to use. We support many protocol versions. (1.8+)

Code for the paper "Extreme Adaptation for Personalized Neural Machine Translation"

This is a plugin to allow OmegaT to source machine translations from Tencent Cloud.

Tools for filtering and cleaning parallel and monolingual corpora for machine translation and other natural language processing tasks.

Suika 🍉 is a Japanese morphological analyzer written in pure Ruby

The aim of this repository is to show a baseline model for text classification by implementing a LSTM-based model coded in PyTorch. In order to provide a better understanding of the model, it will be used a Tweets dataset provided by Kaggle.

C++ Lexer Toolkit Library (LexerTk) https://www.partow.net/programming/lexertk/index.html

Tokenizers and lemmatizers for Go

High-Performance Stemmer, Tokenizer, and Spell Checker for R

Essential NLP & ML, short & fast pure Python code

A morphological analysis library.

Some native scoring script plugins for elasticsearch

A seq2seq with attention dialogue/MT model implemented by TensorFlow.

Curated list of all NLP Resources

The official code repository for MetricMT - a reward optimization method for NMT with learned metrics

Tokenize English sentences using neural networks.

A repository to bind mecab for Python 3.5+. Not using swig nor pybind. (Not Maintained Now)

A tool that locates, downloads, and extracts machine translation corpora

Java wrapper for Instamojo API

Rust re-implementation of OpenFST - library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs). A Python binding is also available.

Get identifiers, paths, URLs and words from the previous command output and use them for the next command in xonsh shell.

Multilingual tokenizer that automatically tags each token with its type

Research code for ACL 2020 paper: "Distilling Knowledge Learned in BERT for Text Generation".

Masakhane Web is a translation web application for solely African Languages.

Training open neural machine translation models

VSCode extension to highlight nested code blocks

TVsub: DCU-Tencent Chinese-English Dialogue Corpus

Lex is an implementation of lex tool in Ruby.

Cross platform C++ library focusing on optimized machine translation on the consumer-grade device.

A simple ruby gem for the DeepL API

A tokenizer for Icelandic text

Machine translation is the task of converting one language to other. Unlike the traditional phrase-based translation system which consists of many small sub-components that are tuned separately, neural machine translation attempts to build and train a single, large neural network that reads a sentence and outputs a correct translation.

OSDG is an open-source tool that maps and connects activities to the UN Sustainable Development Goals (SDGs) by identifying SDG-relevant content in any text. The tool is available online at www.osdg.ai. API access available for research purposes.

R wrappers to connect Python dimensional reduction tools and single cell data objects (Seurat, SingleCellExperiment, etc...)

The Business Scene Dialogue corpus

OpenISS -- a unified multimodal motion data delivery framework.

Vietnamese tokenizer (Maximum Matching and CRF)

Code for Synchronous Bidirectional Neural Machine Translation (SB-NMT)

英中机器文本翻译

A tool for translating text from source grammar to target grammar (context-free) with corresponding dictionary.

A collection of NLP tools for Sinhalese (සිංහල).

📦 Apertium HTTP Server in Python

A simple tokenizer in Ruby for NLP tasks.

Starter code for the VMT task and challenge