399 Open source projects that are alternatives of or similar to Sparkctr

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.

Learn and understand Docker technologies, with real DevOps practice!

Data validation and organization of metadata for data frames and database tables

Scala examples for learning to use Spark

A simplified, lightweight ETL Framework based on Apache Spark

Training of Locally Optimized Product Quantization (LOPQ) models for approximate nearest neighbor search of high dimensional data in Python and Spark.

Java全栈知识架构体系总结

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

The Internals of Spark Structured Streaming

Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.

Moonbox is a DVtaaS (Data Virtualization as a Service) Platform

Qubole Sparklens tool for performance tuning Apache Spark

Essential Spark extensions and helper methods ✨😲

Haskell on Apache Spark.

A cohesive & pragmatic framework of FP centric Scala libraries

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.

An open source framework for building data analytic applications.

Iceberg is a table format for large, slow-moving tabular data

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.

Cross-platform real-time collaboration client optimized for business and organizations.

High Performance HTTP Sidecar Load Balancer

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

Spark Streaming+Flume+Kafka+HBase+Hadoop+Zookeeper实现实时日志分析统计；SpringBoot+Echarts实现数据可视化展示

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

Tools for handling firmwares of DJI products, with focus on quadcopters.

A Scala wrapper for Deeplearning4j, inspired by Keras. Scala + DL + Spark + GPUs

Learning Apache spark,including code and data .Most part can run local.

A Scala feature transformation library for data science and machine learning

Example project implementing best practices for PySpark ETL jobs and applications.

Server for the ListenBrainz project

A tutorial on the most important features and idioms of Scala that you need to use Spark's Scala APIs.

Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition

Scalable genomic data analysis.

Generic Data Ingestion & Dispersal Library for Hadoop

Real Time Analytics and Data Pipelines based on Spark Streaming

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

一个大数据架构师应该掌握的技能

Geo Spatial Data Analytics on Spark

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Building Large-Scale AI Applications for Distributed Big Data

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

DataFusion has now been donated to the Apache Arrow project

This is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.

大数据生态解决方案数据平台：基于大数据、数据平台、微服务、机器学习、商城、自动化运维、DevOps、容器部署平台、数据平台采集、数据平台存储、数据平台计算、数据平台开发、数据平台应用搭建的大数据解决方案。

Kyuubi is a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

大数据博客、笔试题、教程、项目、面经的整理

Optimized Analytics Package for Spark* Platform

The MongoDB Spark Connector

Examples for High Performance Spark

hadoop各组件使用，持续更新

Expressive types for Spark.

电商用户行为分析大数据平台

Alluxio, data orchestration for analytics and machine learning in the cloud

Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL