478 Open source projects that are alternatives of or similar to splink

Resources for tackling record linkage / deduplication / data matching problems

PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.

A list of free data matching and record linkage software.

Learned string similarity for entity names using optimal transport.

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

Welcome to Snowman App – a Data Matching Benchmark Platform.

Merge Dirty Data with Clean Reference Tables

🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

Spark RDD with Lucene's query and entity linkage capabilities

Library and command line utility to do approximate string matching of a source against a bitext index and get matched source and target.

Fuzzy matching and more functionality for spaCy.

Levenshtein algorithm in C

REST job server for Apache Spark

A collection of algorithms for fuzzy search like in Sublime Text.

Cargo with less noise: warnings are skipped until errors are fixed, Neovim integration, etc.

Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

A fuzzy matching string distance library for Scala and Java that includes Levenshtein distance, Jaro distance, Jaro-Winkler distance, Dice coefficient, N-Gram similarity, Cosine similarity, Jaccard similarity, Longest common subsequence, Hamming distance, and more..

Python clone of Spark, a MapReduce alike framework in Python

A Jupyter notebook using some standard techniques for data science and data engineering to analyze data for the 2017 flooding in Houston, TX.

基于Docker构建的Hadoop开发测试环境，包含Hadoop，Hive，HBase，Spark

Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix

☁️ Cloud-scale telemetry ingestion from any stream of data with Azure Event Hubs

R code for Time Series Analysis and Its Applications, Ed 4

fzy inegration with fish. Search history, navigate directories and more. Blazingly fast.

Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources

WhatIs.this: simple entity resolution through Wikipedia

Koalas: pandas API on Apache Spark

Yet Another Dupes Finder

A daily digest of the articles or videos I've found interesting, that I want to share with you.

Examples of spark-lucenerdd

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

machine learning

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

Installations for Data Science. Anaconda, RStudio, Spark, TensorFlow, AWS (Amazon Web Services).

Book recommender system using collaborative filtering based on Spark

Yoyo-leaf is an awesome command-line fuzzy finder.

▁▂▄▆▇█▇▆▄▂▁

Ruby wrapper for Apache Spark

The Internals of Spark SQL

A small package to fuzzy match chinese words

Applying Data Science and Machine Learning to Solve Real World Business Problems

Fuzzy Matching Library for Rust

Apache Spark™ and Scala Workshops

Fuzzy string matching for PHP

OpenRefine reconciliation services for VIAF, ORCID, and Open Library + framework for creating more.

A Spark library for Amazon SageMaker.

Tool for managing data-deduplication within extant compressed archive files, along with a relatively performant BK tree implementation for fuzzy image searching.

📧 CLI to deduplicate mails from mail boxes.

A Spark plugin for reading Excel files via Apache POI

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Web Scraping, Document Deduplication & GPT-2 Fine-tuning with a newly created scam dataset.

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

MLOps Platform

Python machine learning applications in image processing, recommender system, matrix completion, netflix problem and algorithm implementations including Co-clustering, Funk SVD, SVD++, Non-negative Matrix Factorization, Koren Neighborhood Model, Koren Integrated Model, Dawid-Skene, Platt-Burges, Expectation Maximization, Factor Analysis, ISTA, F…

Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.

Spark, Spark Streaming and Spark SQL unit testing strategies