665 Open source projects that are alternatives of or similar to architect_big_data_solutions_with_spark

数据可视化, 数据挖掘, 数据处理 ETL

Testing framework for Databricks notebooks

A simple Spark-powered ETL framework that just works 🍺

This repository is a getting started guide to Singer.

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

In-memory Java DataFrame library

Actively curated list of awesome BI tools. PRs welcome!

Black for Databricks notebooks

A python library to push ethereum blockchain data into an sql database.

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

Monitoring tool to measure flow throughput of data sources and processing components that are part of Data Ingestion and ETL pipelines.

Airbyte is an open-source EL(T) platform that helps you replicate your data in your warehouses, lakes and databases.

The premier open source Data Quality solution

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

A dbt adapter for Databricks.

data wrangling simplicity, complete audit transparency, and at speed

Multivariate data modelling with Copulas in Python

The Frictionless Data website.

csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

A Python package develop for transportation spatio-temporal big data processing, analysis and visualization.

数据质量控制系统

A flexible plotting and data analysis tool.

A python library for calculations perfomed in electronics engineering

Notes and tutorials on how to use python, pandas, seaborn, numpy, matplotlib, scipy for data science.

Curated Python Notebooks for Data Visualization

LabPlot is a FREE, open source and cross-platform Data Visualization and Analysis software accessible to everyone.

一款基于kettle的数据处理web调度控制平台，支持文档资源库和数据库资源库，通过web平台控制kettle数据转换，可作为中间件集成到现有系统中

A scalable general purpose micro-framework for defining dataflows. You can use it to create dataframes, numpy matrices, python objects, ML models, etc.

DBAPI and SQLAlchemy dialect for Databricks Workspace and SQL Analytics clusters

ANTz immersive 3D data visualization engine

Native Dask collection for awkward arrays, and the library to use it.

Compare MLOps Platforms. Breakdowns of SageMaker, VertexAI, AzureML, Dataiku, Databricks, h2o, kubeflow, mlflow...

A distributed block-based data storage and compute engine

A Los Angeles Times analysis of Every shot in Kobe Bryant's NBA career

A collection of Jupyter Notebooks used in conferences or just to have some snippets.

A collection of ML related stuff including notebooks, codes and a curated list of various useful resources such as books and softwares. Almost everything mentioned here is free (as speech not free food) or open-source.

Introduction to Data Science with R (2017)

ETL jobs for Firefox Telemetry

SpectroChemPy is a framework for processing, analyzing and modeling spectroscopic data for chemistry with Python

Repository for the projects needed to complete the Data Analyst Nanodegree.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Python Bitcoin is widely used cryptocurrency for digital market. It is decentralised that means it is not own by government or any other company.Transactions are simple and easy as it doesn’t belong to any country.Records data are stored in Blockchain.Bitcoin price is variable and it is widely used so it is important to predict the price of it f…

ETL for the NASDAQ symbol file

Relational data pipelines for the science lab

Data analysis of IP addresses and networks

CVparser is software for parsing or extracting data out of CV/resumes.

A guide to getting started with Data Science and ML.

An Elasticsearch client tailored to data science workflows.

Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !

🔄 Simple tool for ETL. From SQL to Redis.

DatScan is an initiative to build an open-source CMS that will have the capability to solve any problem using data Analysis just with the help of various modules and a vast standardized module library

A Django app to download, extract and load campaign finance and lobbying activity data from the California Secretary of State's CAL-ACCESS database

Flock: A Low-Cost Streaming Query Engine on FaaS Platforms

heidi : tidy data in Haskell

Synchronization Algorithm Exploration: Techniques to synchronize a SQL database with external destinations.

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

A lib to transform data from jdbc,csv,json to ecah other.

Snorkel - Bootstrap your Data Science

Curso de Ciência da Computação

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes