64 Open source projects that are alternatives of or similar to nomenklatura

A powerful duplicate file finder and an enhanced fork of 'fdupes'.

Accelerating the deduplication and collapsing process for reads with Unique Molecular Identifiers (UMI). Heavily optimized for scalability and orders of magnitude faster than a previous tool.

COSMOS (Causal Oriented Search of Multi-Omic Space) is a method that integrates phosphoproteomics, transcriptomics, and metabolomics data sets.

Extremely fast tool to remove duplicates and other lint from your filesystem

Tool for managing data-deduplication within extant compressed archive files, along with a relatively performant BK tree implementation for fuzzy image searching.

Translator of spreadsheet mappings into R2RML, RML or YARRRML

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

An Efficient RML-Compliant Engine for Knowledge Graph Construction

PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.

Fast, secure, efficient backup program

Make it easier to compare and cross-reference the names of companies and people by applying strong normalisation.

Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix

📖 Generation and Applications of Knowledge Graphs in Systems and Networks Biology

📊 python package for performing deduplication using flexible text matching and cleaning in pandas dataframe

Installer for Thymeflow, a personal knowledge management system.

Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.

Template to deploy a Data Product for data stream processing into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to ingest, provide and create new data assets within the platform.

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.

Resources for tackling record linkage / deduplication / data matching problems

A list of free data matching and record linkage software.

Fast block-level out-of-band BTRFS deduplication tool.

Population Assignment using Genetic, Non-genetic or Integrated Data in a Machine-learning Framework. Methods in Ecology and Evolution. 2018;9:439–446.

Cargo with less noise: warnings are skipped until errors are fixed, Neovim integration, etc.

Find duplicate files

Spark RDD with Lucene's query and entity linkage capabilities

📧 CLI to deduplicate mails from mail boxes.

Building data processing pipelines for documents processing with NLP using Apache NiFi and related services

Record Linkage ToolKit (Find and link entities)

Template to deploy a Data Product for Batch data processing into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to ingest, provide and create new data assets within the platform.

FastCDC implementation in Rust

Powerful RDF Knowledge Graph Generation with [R2]RML Mappings

Simple, configuration-driven backup software for servers and workstations

Upserts, Deletes And Incremental Processing on Big Data.

Data deduplication engine, supporting optional compression and public key encryption.

A bioinformatics API and web-app to integrate multi-omics datasets & interface with public databases.

A toolkit for record linkage and duplicate detection in Python

Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data sc…

Prometheus Alertmanager

WInte.r is a Java framework for end-to-end data integration. The WInte.r framework implements well-known methods for data pre-processing, schema matching, identity resolution, data fusion, and result evaluation.

Dedupe/batch geocode addresses and venues around the world with libpostal

Privacy and Security focused Segment-alternative, in Golang and React

RocketMQ消息幂等去重消费者，支持使用MySQL或者Redis做幂等表，开箱即用

The Common Core Ontology Repository holds the current released version of the Common Core Ontology suite.

Generate duplex/single consensus reads to reduce sequencing noises and remove duplications

Configurable data bridge for permanent ETL jobs

Implementation of Fellegi-Sunter's canonical model of record linkage in Apache Spark, including EM algorithm to estimate parameters

Locality Sensitive Hashing using MinHash in Python/Cython to detect near duplicate text documents

A library for secure, deduplicated, transactional, and verifiable data storage

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Yet Another Dupes Finder

A pair of kernel modules which provide pools of deduplicated and/or compressed block storage.

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

A .NET class library that allows you to import data from different sources into a unified destination

Quickly detect already witnessed data.

Some of the projects i made when starting to learn R for Data Science at the university

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

Reference mapping for single-cell genomics

A Python framework for integrating biological databases and structured data sources in Biological Expression Language (BEL)

Userspace tools for managing VDO volumes.