GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → oeljeklaus-you → LogAnalyzeHelper

oeljeklaus-you / LogAnalyzeHelper

Licence: other
论坛日志分析系统清洗程序(包含IP规则库，UDF开发，MapReduce程序，日志数据)

Programming Languages

java
68154 projects - #9 most used programming language

Labels

hadoop

Projects that are alternatives of or similar to LogAnalyzeHelper

teraslice
Scalable data processing pipelines in JavaScript
Stars: ✭ 48 (+45.45%)
Mutual labels:  hadoop
learning-hadoop-and-spark
Companion to Learning Hadoop and Learning Spark courses on Linked In Learning
Stars: ✭ 146 (+342.42%)
Mutual labels:  hadoop
the-apache-ignite-book
All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above
Stars: ✭ 65 (+96.97%)
Mutual labels:  hadoop
yarn-prometheus-exporter
Export Hadoop YARN (resource-manager) metrics in prometheus format
Stars: ✭ 44 (+33.33%)
Mutual labels:  hadoop
openPDC
Open Source Phasor Data Concentrator
Stars: ✭ 109 (+230.3%)
Mutual labels:  hadoop
HDFS-Netdisc
基于Hadoop的分布式云存储系统 🌴
Stars: ✭ 56 (+69.7%)
Mutual labels:  hadoop
beanszoo
Distributed Java micro-services using ZooKeeper
Stars: ✭ 12 (-63.64%)
Mutual labels:  hadoop
qs-hadoop
大数据生态圈学习
Stars: ✭ 18 (-45.45%)
Mutual labels:  hadoop
bigdata-doc
大数据学习笔记，学习路线，技术案例整理。
Stars: ✭ 37 (+12.12%)
Mutual labels:  hadoop
hive-bigquery-storage-handler
Hive Storage Handler for interoperability between BigQuery and Apache Hive
Stars: ✭ 16 (-51.52%)
Mutual labels:  hadoop
TonY
TonY is a framework to natively run deep learning frameworks on Apache Hadoop.
Stars: ✭ 687 (+1981.82%)
Mutual labels:  hadoop
webhdfs
Node.js WebHDFS REST API client
Stars: ✭ 88 (+166.67%)
Mutual labels:  hadoop
hive to es
同步Hive数据仓库数据到Elasticsearch的小工具
Stars: ✭ 21 (-36.36%)
Mutual labels:  hadoop
gomrjob
gomrjob - a Go Framework for Hadoop Map Reduce Jobs
Stars: ✭ 39 (+18.18%)
Mutual labels:  hadoop
Data-pipeline-project
Data pipeline project
Stars: ✭ 18 (-45.45%)
Mutual labels:  hadoop
JavaFramework
Simple Java Framework,designed for easily develop Spring based java program.Support Bigdata And metadata management.A common elasticsearch comm query tool and so on.
Stars: ✭ 16 (-51.52%)
Mutual labels:  hadoop
smart-data-lake
Smart Automation Tool for building modern Data Lakes and Data Pipelines
Stars: ✭ 79 (+139.39%)
Mutual labels:  hadoop
datalake-etl-pipeline
Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations
Stars: ✭ 39 (+18.18%)
Mutual labels:  hadoop
dockerfiles
Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )
Stars: ✭ 29 (-12.12%)
Mutual labels:  hadoop
iis
Information Inference Service of the OpenAIRE system
Stars: ✭ 16 (-51.52%)
Mutual labels:  hadoop
View All Similar Projects ➔

logAnalyzeHelper

工程的目的

该工程是论坛日志分析系统的辅助工程，主要功能模块是帮助日志分析系统清洗数据、以及在Hive中建立相应的数据表。

工程目录结构

/src

------/main

--------------/java

---------------------/cn.edu.hust

---------------------------------------/preprocess

-------------------------------------------------/domain

-------------------------------------------------------/WebLogBean 日志的POJO对象

-------------------------------------------------/ClickStream r日志清洗数据

---------------------------------------/udf

-------------------------------------------bean

---------------------------------------------Pair IP查找的辅助类

-------------------------------------------BrowserUtils 获取浏览器的UDF函数

-------------------------------------------CityUtils 获取城市的UDF函数类

-------------------------------------------IPUtils 获取省份的UDF函数类

-------------------------------------------OSUtils 获取操作系统UDF函数类

-------------/resources

------/test

------/pom,xml

工程使用的数据集

工程使用的数据集可以在作者的百度云中 下载日志数据,

工程的流程

原始数据--->数据规整--->ETL--->导入mysql数据库--->可视化

ETL使用的SQL

使用的SQL在本本工程中命名为点击流.sql

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].