368 Open source projects that are alternatives of or similar to learning-spark

Cookbook to install Hadoop 2.0+ using Chef

Library for per-file client-side encyption in Hadoop FileSystems such as HDFS or S3.

Hadoop utility to compact small files

🚚 Agile Data Preparation Workflows made easy with dask, cudf, dask_cudf and pyspark

A scalable Apache Hadoop-based implementation of the Pooled Time Series video similarity algorithm based on M. Ryoo et al paper CVPR 2015.

AWS Auto Terminate Idle AWS EMR Clusters Framework is an AWS based solution using AWS CloudWatch and AWS Lambda using a Python script that is using Boto3 to terminate AWS EMR clusters that have been idle for a specified period of time.

Dockerfiles and Docker Compose for HDP 2.6 with Blueprints

Multiple node cluster on Docker for self development.

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Plot live-stats as graph from ApacheSpark application using Lightning-viz

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

10-weeks of technology exploration

A tool and library for easily deploying applications on Apache YARN

学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等

深圳地铁大数据客流分析系统🚇🚄🌟

A book about running Elasticsearch

Teradata Distribution of Presto -- A Distributed SQL Query Engine for Big Data

Data syncing in golang for ClickHouse.

TiDB connectors for Flink/Hive/Presto

TensorBase BE is building a high performance, cloud neutral bigdata warehouse for SMEs fully in Rust.

电商用户行为分析大数据平台

d3 library to build circular graphs

Hive JDBC "uber" or "standalone" jar based on the latest Apache Hive version

High Performance HTTP Sidecar Load Balancer

🍣100本 Java电子书 技术书籍PDF(以下载阅读为荣，以点赞收藏为耻)

DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.

Mirror of Linkedin's Camus

Visualizations of the Hadoop Ecosystem

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.

分布式图数据库 JanusGraph 中文社区，关于 JanusGraph 的一切

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learni…

Code generator for fast log file parsers

定期更新Hadoop生态圈中常用大数据组件文档 重心依次为: Flink Solr Sparksql ES Scala Kafka Hbase/phoenix Redis Kerberos (项目包含hadoop思维导图 印象笔记 Scala版本简单demo 常用工具类 去敏后的train code 持续更新!!!)

Code snippets for solving common big data problems in various platforms. Inspired by Rosetta Code

Node.js WebHDFS REST API client

这是组件库 Jigsaw-七巧板(https://github.com/rdkmaster/jigsaw) 的种子工程，建议所有新增的app都以这个工程作为种子开始构建。

The GIS Tools for Hadoop are a collection of GIS tools for spatial analysis of big data.

Streaming and static data visualization for the modern web.

Big Data Ecosystem Docker

Some class materials for a data processing course using PySpark

编程电子书，电子书，编程书籍，包括C，C#，Docker，Elasticsearch，Git，Hadoop，HeadFirst，Java，Javascript，jvm，Kafka，Linux，Maven，MongoDB，MyBatis，MySQL，Netty，Nginx，Python，RabbitMQ，Redis，Scala，Solr，Spark，Spring，SpringBoot，SpringCloud，TCPIP，Tomcat，Zookeeper，人工智能，大数据类，并发编程，数据库类，数据挖掘，新面试题，架构设计，算法系列，计算机类，设计模式，软件测试，重构优化，等更多分类

学习笔记

Example projects for 'BigInsights for Apache Hadoop' on IBM Bluemix

Workspace for China local user group.

用Python爬取安居客房源信息，并用高德地图进行可视化

Mirror of Apache Hadoop common

Taller SparkR para las Jornadas de Usuarios de R

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Apache Spark Course Material

A robust web archive analytics toolkit

Liquibase extension to add Impala Database support

Big Data Technology Index

Prometheus jmx_exporter configurations for Cloudera Hadoop

A library for manipulating bioinformatics sequencing formats in Apache Spark

论坛日志分析系统清洗程序(包含IP规则库，UDF开发，MapReduce程序，日志数据)

GreyCat - Data Analytics, Temporal data, What-if, Live machine learning

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

Road to Machine Learning

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL