2030 Open source projects that are alternatives of or similar to Dagster

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Beneath is a serverless real-time data platform ⚡️

A site that displays up to date COVID-19 stats, powered by fastpages.

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

PLynx is a domain agnostic platform for managing reproducible experiments and data-oriented workflows.

Example workflows for the drake R package

Projeto de análise de contratos com suspeita de superfaturamento e má qualidade na prestação de serviços.

Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

A tool for building feature stores.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Thain is a distributed flow schedule platform.

Versatile Data Kit (VDK) is an open source framework that enables anybody with basic SQL or Python knowledge to create their own data pipelines.

model-describer : Making machine learning interpretable to humans

Dato Fútbol repository

Actively curated list of awesome BI tools. PRs welcome!

Carefully curated resource links for data science in one place

Data science and Machine Learning GUI programs/ desktop apps with PySimpleGUI package

Apache Superset is a Data Visualization and Data Exploration Platform

BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.

A simple Spark-powered ETL framework that just works 🍺

Docker Apache Airflow

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

Enterprise job scheduling middleware with distributed computing ability.

An easy and fast way to build automation and workflows on Windows, Linux, macOS, and the cloud.

Bitnami Docker Image for Apache Airflow Scheduler

IBIS is a workflow creation-engine that abstracts the Hadoop internals of ingesting RDBMS data.

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Threat Pursuit Virtual Machine (VM): A fully customizable, open-sourced Windows-based distribution focused on threat intelligence analysis and hunting designed for intel and malware analysts as well as threat hunters to get up and running quickly.

The purpose of this project is to share knowledge on how awesome Streamlit is and can be

Verteego Data Suite

The easiest way to automate your data

Introduction to Data Science: A Python Approach to Concepts, Techniques and Applications

ETL with Python - Taught at DWH course 2017 (TAU)

Support content for my blog

An R-focused pipeline toolkit for reproducibility and high-performance computing

A high-level machine learning and deep learning library for the PHP language.

[UNMAINTAINED] Automated machine learning for analytics & production

Accelerate your ML and Data workflows to production. Flyte is a production grade orchestration system for your Data and ML workloads. It has been battle tested at Lyft, Spotify, freenome and others and truly open-source.

IPython widgets, interactive plots, interactive machine learning

Data Science algorithms for Qlik implemented as a Python Server Side Extension (SSE).

Web scrapping and related analytics using Python tools

Example project implementing best practices for PySpark ETL jobs and applications.

📝 An awesome Data Science repository to learn and apply for real world problems.

A convention over configuration workflow orchestrator. Develop locally (Jupyter or your favorite editor), deploy to Airflow or Kubernetes.

Turn complex requirements to workflows without leaving the comfort of your technology stack.

R client for the Elasticsearch HTTP API

The official repository for the AiiDA code

Schedulis is a high performance workflow task scheduling system that supports high availability and multi-tenant financial level features, Linkis computing middleware, and has been integrated into data application development portal DataSphere Studio

大数据采集,抽取平台

Categorical Query Language IDE

Virtual Machine for Desktop Search with Open Semantic Search

A tool for managing builds of monorepo frontend projects with eg. npm- or yarn workspaces, lerna or similar tools into a standalone application - no other tools needed.

Rivery CLI

Machine Learning Platform for Kubernetes (MLOps tools for experimentation and automation)

Reproducible Data Science at Scale!

Career Resources for Data Science, Machine Learning, Big Data and Business Analytics Career Repository

An open source book to learn data science, data analysis and machine learning, suitable for all ages!

⚡ Node.js library to run and orchestrate background jobs with Zenaton Workflow Engine

Example of an ETL Pipeline using Airflow

Dataset format for AI. Build, manage, & visualize datasets for deep learning. Stream data real-time to PyTorch/TensorFlow & version-control it. https://activeloop.ai