632 Open source projects that are alternatives of or similar to versatile-data-kit

Beneath is a serverless real-time data platform ⚡️

Rivery CLI

Easy way to define, execute and store quality rules for your data.

dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.

Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

ELT Code for your Data Warehouse

Example of an ETL Pipeline using Airflow

Live import all your Google Sheets to your data warehouse

The Data Engineering Cookbook

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

A client for connecting and running DDLs on hive metastore.

An orchestration platform for the development, production, and observation of data assets.

The practical use-cases of how to make your Machine Learning Pipelines robust and reliable using Apache Airflow.

Road to Azure Data Engineer Part-I: DP-200 - Implementing an Azure Data Solution

Open Source Feature Flagging and A/B Testing Platform

A schema-aware Scala library for data transformation

Addax is an open source universal ETL tool that supports most of those RDBMS and NoSQLs on the planet, helping you transfer data from any one place to another.

Personal Data Engineering Projects

Python library for building Pangeo Forge recipes.

A python package to create a database on the platform using our moj data warehousing framework

A Pachyderm deep learning tutorial for conference workshops

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

Blog post on ETL pipelines with Airflow

A tool for building feature stores.

NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile y…

[READ-ONLY] PHP - ETL (Extract Transform Load) data processing library

Enterprise-grade, production-hardened, serverless data lake on AWS

Dataform is a framework for managing SQL based data operations in BigQuery, Snowflake, and Redshift

A library for managing, validating, summarizing, and visualizing data.

Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes

Always know what to expect from your data.

A simple Spark-powered ETL framework that just works 🍺

Free and open source schema versioning and database migration made natively with .NET Core.

Web-based SQL editor run in your own private cloud. Supports MySQL, Postgres, SQL Server, Vertica, Crate, ClickHouse, Trino, Presto, SAP HANA, Cassandra, Snowflake, BigQuery, SQLite, and more with ODBC

DE or DIE meetup made by data engineers for data engineers. Currently in Russian only.

Powerful RDF Knowledge Graph Generation with [R2]RML Mappings

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

Example project implementing best practices for PySpark ETL jobs and applications.

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

Fancy stream processing made operationally mundane

Python based Wikidata framework for easy dataframe extraction

Starlake is a Spark Based On Premise and Cloud ELT/ETL Framework for Batch & Stream Processing

An Elasticsearch client tailored to data science workflows.

A curated list of awesome big data frameworks, ressources and other awesomeness.

Metamapper is a data discovery and documentation platform for improving how teams understand and interact with their data.

Finally, an MMORPG that's all about grinding and doing chores.

☁️ nodejs kafka connect connector for Google BigQuery

Connection-Pooling Compile-Time ORM for Nim

A COBOL parser and Mainframe/EBCDIC data source for Apache Spark

code, labs and lectures for the course

An SQLite3 driver for Elixir

Loadable spellfix1 extension for sqlite as python package

A CLI and library to run Singer Taps and Targets

Lightweight SQLite editor for Windows

Sqlite3 command line in a docker container

Code relating to scraping public police data.