366 Open source projects that are alternatives of or similar to hadoop-etl-udfs

An Eyeriss Chip (researched by MIT, a CNN accelerator) simulator and New DNN framework "Hive"

Computes and visualizes the sentiment analysis of tweets of US States in real-time using Storm.

This repository contains Machine-Learning MapReduce codes for Hadoop which are written from scratch (without using any package or library). E.g. Prediction (Linear and Logistic Regression), Clustering (K-Means), Classification (KNN) etc.

Albis: High-Performance File Format for Big Data Systems

Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

Apache Phoenix / Hbase Spring Boot Microservices

Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.

Library for per-file client-side encyption in Hadoop FileSystems such as HDFS or S3.

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

Library to read a subset of Parquet files

后端开发常用框架文档及中文翻译，包含 Spring 系列文档（Spring, Spring Boot, Spring Cloud, Spring Security, Spring Session），大数据（Apache Hive, HBase, Apache Flume），日志（Log4j2, Logback），Http Server（NGINX，Apache），Python，数据库（OpenTSDB，MySQL，PostgreSQL）等最新官方文档以及对应的中文翻译。

Docker containers for Hadoop.

Visualizations of the Hadoop Ecosystem

基于spark的外卖大数据平台分析系统

Manipulate arrays of complex data structures as easily as Numpy.

Provides Mysql UDF commands to synchronize data from Mysql to Redis.

A SQLite vtable extension to read Parquet files

Hive API server (offloads most API calls from hived) implemented using Python+SQL

A collection of Apache Parquet add-on modules

Spark SQL index for Parquet tables

Demonstration of a Hive Input Format for Iceberg

Docker image for main Apache Hadoop components (Yarn/Hdfs)

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

tools for bigData

Make record oriented data to columnar format.

python ETL framework

winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

GCS support for avro-tools, parquet-tools and protobuf

Quilt is a self-organizing data hub for S3

An Open Source unit test framework for Hive queries based on JUnit 4 and 5

Parquet file generator

电商用户行为分析大数据平台

High performance distributed data processing engine

基于Hadoop的分布式云存储系统 🌴

Optimized Analytics Package for Spark* Platform

DataX 是异构数据广泛使用的离线数据同步工具/平台，实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

A facebook for data

TonY is a framework to natively run deep learning frameworks on Apache Hadoop.

Fast data store for Pandas time-series data

A tool for data sampling, data generation, and data diffing

Tidy up Spark and Hadoop tutorials.

🍣100本 Java电子书 技术书籍PDF(以下载阅读为荣，以点赞收藏为耻)

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Efficient training of deep recommenders on cloud.

A collection of pentest tools and resources targeting Hadoop environments

异构存储数据迁移

Lightweight and blazing fast key-value database written in pure Dart.

GraphQL service for arrow tables and parquet data sets.

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

A custom extractor designed to read parquet for Azure Data Lake Analytics

Easy parsing of Apache HTTPD and NGINX access logs with Java, Hadoop, Hive, Pig, Flink, Beam, Storm, Drill, ...

Apache Calcite

Service for automatically managing and cleaning up unreferenced data

A high-performance, scalable and efficient ShuffleManager plugin for Apache Spark, utilizing UCX communication layer

DBAPI and SQLAlchemy dialect for Databricks Workspace and SQL Analytics clusters

A tool and library for easily deploying applications on Apache YARN

大数据生态圈学习

gomrjob - a Go Framework for Hadoop Map Reduce Jobs

Distributed Deep Learning, with a focus on distributed training, using Keras and Apache Spark.