GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → haozhang-x → spark-waimai

haozhang-x / spark-waimai

Licence: other
基于spark的外卖大数据平台分析系统

Programming Languages

scala
5932 projects
python
139335 projects - #7 most used programming language
shell
77523 projects

Labels

sparkhive

Projects that are alternatives of or similar to spark-waimai

simple-ddl-parser
Simple DDL Parser to parse SQL (HQL, TSQL, AWS Redshift, BigQuery, Snowflake and other dialects) ddl files to json/python dict with full information about columns: types, defaults, primary keys, etc. & table properties, types, domains, etc.
Stars: ✭ 76 (+216.67%)
Mutual labels:  hive
HiveJdbcStorageHandler
No description or website provided.
Stars: ✭ 21 (-12.5%)
Mutual labels:  hive
web-click-flow
网站点击流离线日志分析
Stars: ✭ 14 (-41.67%)
Mutual labels:  hive
databricks-dbapi
DBAPI and SQLAlchemy dialect for Databricks Workspace and SQL Analytics clusters
Stars: ✭ 21 (-12.5%)
Mutual labels:  hive
hive-cube
Data self exporting and monitoring platform based on Hive data warehouse. https://hc.smartloli.org
Stars: ✭ 34 (+41.67%)
Mutual labels:  hive
apiary
Apiary provides modules which can be combined to create a federated cloud data lake
Stars: ✭ 30 (+25%)
Mutual labels:  hive
xxhadoop
Data Analysis Using Hadoop/Spark/Storm/ElasticSearch/MachineLearning etc. This is My Daily Notes/Code/Demo. Don't fork, Just star !
Stars: ✭ 37 (+54.17%)
Mutual labels:  hive
hivemind
Hive API server (offloads most API calls from hived) implemented using Python+SQL
Stars: ✭ 46 (+91.67%)
Mutual labels:  hive
hive-jdbc-driver
An alternative to the "hive standalone" jar for connecting Java applications to Apache Hive via JDBC
Stars: ✭ 31 (+29.17%)
Mutual labels:  hive
BigDataTools
tools for bigData
Stars: ✭ 36 (+50%)
Mutual labels:  hive
beekeeper
Service for automatically managing and cleaning up unreferenced data
Stars: ✭ 43 (+79.17%)
Mutual labels:  hive
liquibase-impala
Liquibase extension to add Impala Database support
Stars: ✭ 23 (-4.17%)
Mutual labels:  hive
DaFlow
Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (+0%)
Mutual labels:  hive
hadoopoffice
HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)
Stars: ✭ 56 (+133.33%)
Mutual labels:  hive
hiveberg
Demonstration of a Hive Input Format for Iceberg
Stars: ✭ 22 (-8.33%)
Mutual labels:  hive
data-profiling
a set of scripts to pull meta data and data profiling metrics from relational database systems
Stars: ✭ 57 (+137.5%)
Mutual labels:  hive
aaocp
一个对用户行为日志进行分析的大数据项目
Stars: ✭ 53 (+120.83%)
Mutual labels:  hive
ETL-Starter-Kit
📁 Extract, Transform, Load (ETL) 👷 refers to a process in database usage and especially in data warehousing. This repository contains a starter kit featuring ETL related work.
Stars: ✭ 21 (-12.5%)
Mutual labels:  hive
cobra-policytool
Manage Apache Atlas and Ranger configuration for your Hadoop environment.
Stars: ✭ 16 (-33.33%)
Mutual labels:  hive
waggle-dance
Hive federation service. Enables disparate tables to be concurrently accessed across multiple Hive deployments.
Stars: ✭ 194 (+708.33%)
Mutual labels:  hive
View All Similar Projects ➔

网易Spark大数据开发工程师微专业实战大项目
基于Spark的外卖大数据平台分析系统

说明

可将你的hive-site.xml文件放到resources下， Spark应用程序在IDEA里面运行的时候，会读取这个Hive的配置文件，加载相应的Hive配置信息

运行顺序td→ti→tf→tm→ta

package说明

  1. ta为应用层
    • 可直接运行生成对应的表
  2. td为维度层
    • 可直接运行生成对应的表
  3. tf为整合层
    • 可直接运行生成对应的表
  4. tm为中间层
    • 可直接运行生成对应的表
  5. ti为接口层
    • 可直接运行生成对应的表

后续可用Apache Kylin，来提高SQL执行的效率
下面展示一些Kylin的WebUI的可视化效果

图1 图1 图2 图2 图2 图3

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].