Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

基于spark的外卖大数据平台分析系统

Today I Learned

tools for bigData

Hive API server (offloads most API calls from hived) implemented using Python+SQL

와이빅타 엔지니어링팀의 자료를 정리해두는 곳입니다.

一个对用户行为日志进行分析的大数据项目

Open-source distribute workflow schedule tools, also support streaming task.

📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.

DLUX distributed deterministic finite state automata. Built for HIVE to take advantage of free transactions using multi-sig and escrow for security.

网站点击流离线日志分析

Manage Apache Atlas and Ranger configuration for your Hadoop environment.

最好的大数据项目。《Titan数据运营系统》，本项目是一个全栈闭环系统，我们有用作数据可视化的web系统，然后用flume-kafaka-flume进行日志的读取，在hive设计数仓，编写spark代码进行数仓表之间的转化以及ads层表到mysql的迁移，使用azkaban进行定时任务的调度，使用技术：Java/Scala语言，Hadoop、Spark、Hive、Kafka、Flume、Azkaban、SpringBoot，Bootstrap， Echart等；

Hive federation service. Enables disparate tables to be concurrently accessed across multiple Hive deployments.

A naive solution to document schemas

Apiary provides modules which can be combined to create a federated cloud data lake

云计算之hadoop、hive、hue、oozie、sqoop、hbase、zookeeper环境搭建及配置文件

Mutation testing framework and code coverage for Hive SQL

基于开源Litemall电商项目的大数据项目，包含前端埋点(openresty+lua)、后端埋点；数据仓库(五层)、实时计算和用户画像。大数据平台采用CDH6.3.2(已使用vagrant+ansible脚本化)，同时也包含了Azkaban的workflow。

Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive