1453 Open source projects that are alternatives of or similar to aut

Fundamentals of Spark with Python (using PySpark), code examples

Simple and Distributed Machine Learning

简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

RDMA accelerated, high-performance, scalable and efficient ShuffleManager plugin for Apache Spark

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Structural Bioinformatics Training Workshop & Hackathon 2018

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

A collection of tutorials on Hadoop, MapReduce, Spark, Docker

Big Data Processing Framework - Unified Data API or SQL on Any Storage

Routines and data structures for using isarn-sketches idiomatically in Apache Spark

Koalas: pandas API on Apache Spark

PySpark Algorithms Book: https://www.amazon.com/dp/B07X4B2218/ref=sr_1_2

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

Apache Spark Connector for Azure Cosmos DB

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Simple and Distributed Machine Learning

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

C# and F# language binding and extensions to Apache Spark

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

A complete (distributed) BigData stack, running in containers

PySpark Cheat Sheet - example code to help you learn PySpark and develop apps faster

Griffon Data Science Virtual Machine

A large-scale entity and relation database supporting aggregation of properties

Basin is a visual programming editor for building Spark and PySpark pipelines. Easily build, debug, and deploy complex ETL pipelines from your browser

大数据入门指南 ⭐

A Spark cluster setup running on Docker containers

Apache Spark on AWS Lambda

A Clojure dataframe library that runs on Spark

Manipulate arrays of complex data structures as easily as Numpy.

An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.

Apache Calcite

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

open source tools for interaction with IBM PAIRS:

No description or website provided.

Social Network Facebook Analysis (Python, Networkx)

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …

Methods for the parallel and distributed analysis and mining of the Protein Data Bank using MMTF and Apache Spark.

curated list of awesome tools and libraries for specific domains

Companion to Learning Hadoop and Learning Spark courses on Linked In Learning

Code Library for My Blog

Sentiment Analysis of a Twitter Topic with Spark Structured Streaming

Information Inference Service of the OpenAIRE system

Apache Spark in your IDE with gradle

OLAP and ETL of Big Data

low-level helpers for Apache Spark libraries and tests

Pyspark in Google Colab: A simple machine learning (Linear Regression) model

GeoTrellis for PySpark

Real-world Spark pipelines examples

Automatic feature engineering using Generative Adversarial Networks using Deeplearning4j and Apache Spark.

Powerful rapid automatic EDA and feature engineering library with a very easy to use API 🌟

Bulletproof Apache Spark jobs with fast root cause analysis of failures.

JupyterLab extension that enables monitoring launched Apache Spark jobs from within a notebook

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

ACID Data Source for Apache Spark based on Hive ACID

Apache Arrow DataFusion SQL Query Engine

基于Spark2.2新闻网大数据实时系统项目