indix / Whatthelang
Licence: apache-2.0
Lightning Fast Language Prediction 🚀
Stars: ✭ 130
Programming Languages
Projects that are alternatives of or similar to Whatthelang
spacy-fastlang
Language detection using Spacy and Fasttext
Stars: ✭ 34 (-73.85%)
Mutual labels: language-detection, fasttext
Pytorchtext
1st Place Solution for Zhihu Machine Learning Challenge . Implementation of various text-classification models.(知乎看山杯第一名解决方案)
Stars: ✭ 1,022 (+686.15%)
Mutual labels: fasttext
React Native Localize
🌍 A toolbox for your React Native app localization
Stars: ✭ 1,682 (+1193.85%)
Mutual labels: language-detection
Nlp Journey
Documents, papers and codes related to Natural Language Processing, including Topic Model, Word Embedding, Named Entity Recognition, Text Classificatin, Text Generation, Text Similarity, Machine Translation),etc. All codes are implemented intensorflow 2.0.
Stars: ✭ 1,290 (+892.31%)
Mutual labels: fasttext
Spacy Cld
Language detection extension for spaCy 2.0+
Stars: ✭ 103 (-20.77%)
Mutual labels: language-detection
Embeddingsviz
Visualize word embeddings of a vocabulary in TensorBoard, including the neighbors
Stars: ✭ 40 (-69.23%)
Mutual labels: fasttext
Guess Language.el
Emacs minor mode that detects the language you're typing in. Automatically switches spell checker. Supports multiple languages per document.
Stars: ✭ 78 (-40%)
Mutual labels: language-detection
Google Translate Php
🌐 Free Google Translate API PHP Package. Translates totally free of charge.
Stars: ✭ 1,131 (+770%)
Mutual labels: language-detection
Fasttext.py
A Python interface for Facebook fastText
Stars: ✭ 1,091 (+739.23%)
Mutual labels: fasttext
Nlp Models Tensorflow
Gathers machine learning and Tensorflow deep learning models for NLP problems, 1.13 < Tensorflow < 2.0
Stars: ✭ 1,603 (+1133.08%)
Mutual labels: language-detection
Cvpr paper search tool
Automatic paper clustering and search tool by fastext from Facebook Research
Stars: ✭ 43 (-66.92%)
Mutual labels: fasttext
Half Size
Code for "Effective Dimensionality Reduction for Word Embeddings".
Stars: ✭ 89 (-31.54%)
Mutual labels: fasttext
Tgcontest
Telegram Data Clustering contest solution by Mindful Squirrel
Stars: ✭ 74 (-43.08%)
Mutual labels: fasttext
whatthelang
Lightning Fast Language Prediction 🚀
Dependencies
The dependencies can be installed using the requirements.txt file:
$ pip install -r requirements.txt
Install
$ pip install whatthelang
Basic Usage
Predicting Language using whatthelang
>>> from whatthelang import WhatTheLang
>>> wtl = WhatTheLang()
>>> wtl.predict_lang("Mother")
'en'
>>> wtl.predict_lang("தாய்")
'ta'
>>> wtl.predict_lang("അമ്മ")
'ml'
>>> wtl.predict_lang("पिता")
'hi'
Batch Prediction is also supported
>>>wtl.predict_lang(["അമ്മ","पिता","teacher"])
['ml','hi','en']
Supported Languages
Supports 176 languages . The ISO codes for the corresponding languages are as below.
af als am an ar arz as ast av az azb ba bar bcl be bg bh bn bo bpy br bs bxr ca cbk
ce ceb ckb co cs cv cy da de diq dsb dty dv el eml en eo es et eu fa fi fr frr fy ga
gd gl gn gom gu gv he hi hif hr hsb ht hu hy ia id ie ilo io is it ja jbo jv ka kk km
kn ko krc ku kv kw ky la lb lez li lmo lo lrc lt lv mai mg mhr min mk ml mn mr mrj ms
mt mwl my myv mzn nah nap nds ne new nl nn no oc or os pa pam pfl pl pms pnb ps pt qu
rm ro ru rue sa sah sc scn sco sd sh si sk sl so sq sr su sv sw ta te tg th tk tl tr
tt tyv ug uk ur uz vec vep vi vls vo wa war wuu xal xmf yi yo yue zh
Model Training Details
Quantized model built using Fasttext. More details present in the fasttext blog
Reference
whatthelang
is powered by FastText
Enriching Word Vectors with Subword Information
[1] P. Bojanowski*, E. Grave*, A. Joulin, T. Mikolov, Enriching Word Vectors with Subword Information
@article{bojanowski2016enriching,
title={Enriching Word Vectors with Subword Information},
author={Bojanowski, Piotr and Grave, Edouard and Joulin, Armand and Mikolov, Tomas},
journal={arXiv preprint arXiv:1607.04606},
year={2016}
}
Bag of Tricks for Efficient Text Classification
[2] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, Bag of Tricks for Efficient Text Classification
@article{joulin2016bag,
title={Bag of Tricks for Efficient Text Classification},
author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Mikolov, Tomas},
journal={arXiv preprint arXiv:1607.01759},
year={2016}
}
FastText.zip: Compressing text classification models
[3] A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jégou, T. Mikolov, FastText.zip: Compressing text classification models
@article{joulin2016fasttext,
title={FastText.zip: Compressing text classification models},
author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Douze, Matthijs and J{\'e}gou, H{\'e}rve and Mikolov, Tomas},
journal={arXiv preprint arXiv:1612.03651},
year={2016}
}
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].