665 Open source projects that are alternatives of or similar to architect_big_data_solutions_with_spark

Česká otevřená data

📊https://datav.io is a modern APM, provide observability for your business, application and infrastructure. It's also a lightweight alternative to Grafana.

A collection of ML related stuff including notebooks, codes and a curated list of various useful resources such as books and softwares. Almost everything mentioned here is free (as speech not free food) or open-source.

Data Analysis with Bootstrapped ESTimation

Udacity Data Engineering Nano Degree (DEND)

Data reporting and visualization for your app

Example of integrating & using Amazon Textract, Amazon Comprehend, Amazon Comprehend Medical, Amazon Kendra to automate the processing of documents for use cases such as enterprise search and discovery, control and compliance, and general business process workflow.

Streamlit — The fastest way to build data apps in Python

(Spatial) data harmonisation with hale studio (formerly HUMBOLDT Alignment Editor)

Introduction to Data Science with R (2017)

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

Code relating to scraping public police data.

Create your free account at runalyze.com

Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Immutable and grow-only Pandas-like DataFrames with a more explicit and consistent interface.

Shell script to run OpenRefine in batch mode (import, transform, export). It orchestrates OpenRefine (server) and a python client that communicates with the OpenRefine API.

A curated list of resources for learning bioinformatics.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

ESC Team's scorecard tools

ETL jobs for Firefox Telemetry

Manipulate JSON-like data with NumPy-like idioms.

A luigi powered analytics / warehouse stack

Frontend framework for rapid data (JSON) analysis, sharable serverless reports and dashboards

ETL for the IEX Stocks API

📖 每一个伟大的思想和行动都有一个微不足道的开始

A Singer.io Target for Postgres

SpectroChemPy is a framework for processing, analyzing and modeling spectroscopic data for chemistry with Python

Easy to use Python library of customized functions for cleaning and analyzing data.

ETL with Python - Taught at DWH course 2017 (TAU)

Graphical tool for data manipulation written in C++/Qt

基于spark-ml,spark-mllib,spark-streaming的推荐算法实现

GoAccess is a real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

ETLy is an add-on dashboard service on top of Apache Airflow.

Advanced and Fast Data Transformation in R

Repository for the projects needed to complete the Data Analyst Nanodegree.

📖 [译] Matplotlib 用户指南

PHP ETL (Extract / Transform / Load) library with SOLID principles + almost no dependency.

A model-driven dynamically-configurable framework to acquire data from external sources and save it to your database.

Configurable data bridge for permanent ETL jobs

Extensive and accessible COVID-19 data + forecasting for counties and hospitals. 📈

Public, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores

Data Analysis with Bootstrap Estimation in R

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Countly Product Analytics SDK for websites and web applications

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

🚀 打印设计、可视化、大屏、编辑器、设计器、数据分析、报表设计、组件化、表单设计、h5页面、调查问卷、pdf生成、流程图、试卷、SVG、图形元素、物联网

🐧基于spark streaming+flume+kafka+hbase的实时日志处理分析系统(分为控制台版本和基于springboot、Echarts等的Web UI可视化版本)

Python package for consolidated and extensive Univariate,Bivariate Data Analysis and Visualization catering to both categorical and continuous datasets.

AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.

Crawler for Fundamental analysis platform for BOVESPA stocks, generating a score for each share according to the selected criteria on the indicators.

Python Bitcoin is widely used cryptocurrency for digital market. It is decentralised that means it is not own by government or any other company.Transactions are simple and easy as it doesn’t belong to any country.Records data are stored in Blockchain.Bitcoin price is variable and it is widely used so it is important to predict the price of it f…

The backoffice web application of d:swarm (https://github.com/dswarm/dswarm-documentation/wiki)

A CLI and library to run Singer Taps and Targets

The power of Chart.js with Python

金融市场与体育彩券市场 --- 数据分析与量化交易

The Human Neocortical Neurosolver (HNN) is a software tool that gives researchers/clinicians the ability to develop/test hypotheses on circuit mechanisms underlying EEG/MEG data.

Data ETL & Analysis on the global and Mexican datasets of the COVID-19 pandemic.

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Bender - Serverless ETL Framework

bqloader is a simple ETL framework to load data from Cloud Storage into BigQuery.