fayson / Cdhproject

Real Time Analytics and Data Pipelines based on Spark Streaming

DataFusion has now been donated to the Apache Arrow project

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

A tutorial on the most important features and idioms of Scala that you need to use Spark's Scala APIs.

The MongoDB Spark Connector

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

Geo Spatial Data Analytics on Spark

Expressive types for Spark.

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

电商用户行为分析大数据平台

Essential Spark extensions and helper methods ✨😲

Alluxio, data orchestration for analytics and machine learning in the cloud

A cohesive & pragmatic framework of FP centric Scala libraries

Training of Locally Optimized Product Quantization (LOPQ) models for approximate nearest neighbor search of high dimensional data in Python and Spark.

Use Jupyter Notebooks to demonstrate how to build a Recommender with Apache Spark & Elasticsearch

An open source framework for building data analytic applications.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Scalable genomic data analysis.

Example project implementing best practices for PySpark ETL jobs and applications.