RickyHuo / Hangout Output Clickhouse
Implementation of ClickHouse output for hangout
Stars: ✭ 86
Programming Languages
java
68154 projects - #9 most used programming language
Labels
Projects that are alternatives of or similar to Hangout Output Clickhouse
Clickhouse Presentations
Presentations, meetups and talks about ClickHouse
Stars: ✭ 483 (+461.63%)
Mutual labels: clickhouse
Analytics
Simple, open-source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics.
Stars: ✭ 9,469 (+10910.47%)
Mutual labels: clickhouse
Logstash Output Clickhouse
Implementation of ClickHouse output for logstash
Stars: ✭ 75 (-12.79%)
Mutual labels: clickhouse
Clickhouse Driver
ClickHouse Python Driver with native interface support
Stars: ✭ 562 (+553.49%)
Mutual labels: clickhouse
Homebrew Clickhouse
ClickHouse for MacOS Sierra and High Sierra.
Stars: ✭ 50 (-41.86%)
Mutual labels: clickhouse
Clickhouse Grafana
Clickhouse datasource for grafana
Stars: ✭ 462 (+437.21%)
Mutual labels: clickhouse
Loghouse
Ready to use log management solution for Kubernetes storing data in ClickHouse and providing web UI.
Stars: ✭ 805 (+836.05%)
Mutual labels: clickhouse
Clickhouse Operator
The ClickHouse Operator creates, configures and manages ClickHouse clusters running on Kubernetes
Stars: ✭ 478 (+455.81%)
Mutual labels: clickhouse
Gorose
GoRose(go orm), a mini database ORM for golang, which inspired by the famous php framwork laravle's eloquent. It will be friendly for php developer and python or ruby developer. Currently provides six major database drivers: mysql,sqlite3,postgres,oracle,mssql, Clickhouse.
Stars: ✭ 947 (+1001.16%)
Mutual labels: clickhouse
Flow Pipeline
A set of tools and examples to run a flow-pipeline (sFlow, NetFlow)
Stars: ✭ 86 (+0%)
Mutual labels: clickhouse
Clickhouse Scala Client
Clickhouse Scala Client with Reactive Streams support
Stars: ✭ 84 (-2.33%)
Mutual labels: clickhouse
Clickhouse Jdbc Bridge
A JDBC proxy from ClickHouse to external databases
Stars: ✭ 69 (-19.77%)
Mutual labels: clickhouse
Hangout output plugin : Clickhouse
- Author: rickyHuo
- Homepage: https://github.com/RickyHuo/hangout-output-clickhouse
- Version: 0.0.8
Description
使用Hangout将数据清洗写入ClickHouse,兼容hangout-dist-0.3.0-release, 0.4.0版本hangout兼容性未知。
如果线上有现成的Spark环境,可以使用Waterdrop接入数据,支持Kafka、HDFS、Hive等。
Options
name | type | required | default value |
---|---|---|---|
bulk_size | int | no | 1000 |
database | string | yes | - |
fields | list | yes | - |
format | string | no | TabSeparated |
host | string | yes | - |
table | string | yes | - |
password | string | yes | - |
username | string | yes | - |
bulk_size [number]
批次写入量,默认为1000,当且仅当数据条数达到bulk_size时,才会触发写入操作。
dev版本正在开发时间触发器,敬请期待...
database [string]
ClickHouse database.
fields [list]
期望写入ClickHouse的数据字段。
format [string]
数据插入格式ClickHouse Format Introduction
当前支持JSONEachRow
以及TabSeparated
。
host [string]
ClickHouse集群地址,多个以逗号","隔开。
table [string]
ClickHouse table name.
username [string]
ClickHouse withCredit username.
password [string]
ClickHouse withCredit password.
Examples
outputs:
- com.sina.bip.hangout.outputs.Clickhouse:
host: localhost:8123
username: user
password: passwd
database: apm
table: apm_netdiagno
fields: ['_device_id', '_ping_small', '_domain', '_traceroute', '_ping_big', 'date', 'ts', '_snet']
bulk_size: 500
使用
Tabseparated
(default)将fields中对应的字段写入ClickHouse。
outputs:
- com.sina.bip.hangout.outputs.Clickhouse:
host: localhost:8123
username: user
password: passwd
database: apm
format: JSONEachRow
table: apm_netdiagno
bulk_size: 500
使用
JSONEachRow
将数据写入ClickHouse,使用时务必保证清洗后的数据没有多余的字段且与表结构对应。使用JSONEachRow
则不需要配置fields
参数。
Tips
在写入ClickHouse之前,Date和DateTime类型的字段需要转换为指定格式。
-
Date
yyyy-MM-dd
-
DateTime
yyyy-MM-dd HH:mm:ss
-
UNIX
例如:1533535518
Supported data types
- UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64
- Float32, Float64
- Date
- DateTime
- Array(String)
- LowCardinality(T)
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].