All Projects → tidb-incubator → TiBigData

tidb-incubator / TiBigData

Licence: Apache-2.0 license
TiDB connectors for Flink/Hive/Presto

Programming Languages

java
68154 projects - #9 most used programming language
ANTLR
299 projects

Projects that are alternatives of or similar to TiBigData

bigdata-doc
大数据学习笔记,学习路线,技术案例整理。
Stars: ✭ 37 (-80.73%)
Mutual labels:  hive, bigdata, flink
dpkb
大数据相关内容汇总,包括分布式存储引擎、分布式计算引擎、数仓建设等。关键词:Hadoop、HBase、ES、Kudu、Hive、Presto、Spark、Flink、Kylin、ClickHouse
Stars: ✭ 123 (-35.94%)
Mutual labels:  presto, hive, flink
hadoopoffice
HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)
Stars: ✭ 56 (-70.83%)
Mutual labels:  hive, bigdata, flink
Hadoopcryptoledger
Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive
Stars: ✭ 126 (-34.37%)
Mutual labels:  hive, bigdata, flink
dockerfiles
Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )
Stars: ✭ 29 (-84.9%)
Mutual labels:  hive, bigdata, flink
God Of Bigdata
专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+3029.17%)
Mutual labels:  hive, bigdata, flink
Bigdataguide
大数据学习,从零开始学习大数据,包含大数据学习各阶段学习视频、面试资料
Stars: ✭ 817 (+325.52%)
Mutual labels:  hive, bigdata, flink
Bigdata Notes
大数据入门指南 ⭐
Stars: ✭ 10,991 (+5624.48%)
Mutual labels:  hive, bigdata
Quicksql
A Flexible, Fast, Federated(3F) SQL Analysis Middleware for Multiple Data Sources
Stars: ✭ 1,821 (+848.44%)
Mutual labels:  hive, flink
Bigdata practice
大数据分析可视化实践
Stars: ✭ 166 (-13.54%)
Mutual labels:  hive, bigdata
incubator-linkis
Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB...), exposes various interfaces(REST, JDBC, Java ...), with multi-tenancy, high performance, and resource control.
Stars: ✭ 2,459 (+1180.73%)
Mutual labels:  presto, hive
Trino
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Stars: ✭ 4,581 (+2285.94%)
Mutual labels:  presto, hive
Yanagishima
Web UI for Trino, Presto, Hive, Elasticsearch, SparkSQL
Stars: ✭ 424 (+120.83%)
Mutual labels:  presto, hive
Repository
个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。
Stars: ✭ 92 (-52.08%)
Mutual labels:  hive, flink
Hops Examples
Examples for Deep Learning/Feature Store/Spark/Flink/Hive/Kafka jobs and Jupyter notebooks on Hops
Stars: ✭ 84 (-56.25%)
Mutual labels:  hive, flink
Dataspherestudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
Stars: ✭ 1,195 (+522.4%)
Mutual labels:  hive, flink
Apache Spark Hands On
Educational notes,Hands on problems w/ solutions for hadoop ecosystem
Stars: ✭ 74 (-61.46%)
Mutual labels:  hive, bigdata
Haproxy Configs
80+ HAProxy Configs for Hadoop, Big Data, NoSQL, Docker, Elasticsearch, SolrCloud, HBase, MySQL, PostgreSQL, Apache Drill, Hive, Presto, Impala, Hue, ZooKeeper, SSH, RabbitMQ, Redis, Riak, Cloudera, OpenTSDB, InfluxDB, Prometheus, Kibana, Graphite, Rancher etc.
Stars: ✭ 106 (-44.79%)
Mutual labels:  presto, hive
Pyhive
Python interface to Hive and Presto. 🐝
Stars: ✭ 1,378 (+617.71%)
Mutual labels:  presto, hive
Cube.js
📊 Cube — Open-Source Analytics API for Building Data Apps
Stars: ✭ 11,983 (+6141.15%)
Mutual labels:  presto, hive

TiBigData

License


EN doc CN doc

Misc BigData components for TiDB, Presto, Flink and MapReduce connectors for example.

License

TiBigData project is under the Apache 2.0 license. See the LICENSE file for details.

Code style

TiBigData use Google Code Style.

Format the code with the following command:

mvn com.coveo:fmt-maven-plugin:format

Getting Started

Compiling TiBigData requires git and downloading source code directly is not supported.

Flink-TiDB-Connector

PrestoSQL-TiDB-Connector

Trino-TiDB-Connector

PrestoDB-TiDB-Connector

MapReduce-TiDB-Connector

Hive-TiDB-Storage-Handler

Run Tests

Use the following command to run the integration test.

export TIDB_HOST="127.0.0.1"
export TIDB_PORT="4000"
export TIDB_USER="root"
export TIDB_PASSWORD=""
mvn clean test-compile failsafe:integration-test -am -pl ${MODULE_NAME}

Use the following command to run the unit test.

mvn clean test -am -pl ${MODULE_NAME}

Write integration test

First, make sure the module add dependency:

 <dependency>
      <groupId>io.tidb</groupId>
      <artifactId>bigdata-test</artifactId>
      <version>${project.version}</version>
      <scope>test</scope>
</dependency>

Second, use JUnit Category to clarify integration-test:

import io.tidb.bigdata.test.IntegrationTest;

@Category(IntegrationTest.class)
public class ConnectorsPluginTest {
}

Community

Lark / Feishu Group:

Supported by

YourKit

YourKit Logo

YourKit supports open source projects with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of YourKit Java Profiler, YourKit .NET Profiler, and YourKit YouMonitor.

IntelliJ IDEA

IntelliJ IDEA is a Java integrated development environment (IDE) for developing computer software.
It is developed by JetBrains (formerly known as IntelliJ), and is available as an Apache 2 Licensed community edition,
and in a proprietary commercial edition. Both can be used for commercial development.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].