All Projects → Arvin-Mark → DataX-src

Arvin-Mark / DataX-src

Licence: other
DataX 是异构数据广泛使用的离线数据同步工具/平台,实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

Programming Languages

java
68154 projects - #9 most used programming language
python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to DataX-src

Addax
Addax is an open source universal ETL tool that supports most of those RDBMS and NoSQLs on the planet, helping you transfer data from any one place to another.
Stars: ✭ 615 (+2828.57%)
Mutual labels:  hive, etl, oracle, sqlserver, datax
Pyetl
python ETL framework
Stars: ✭ 33 (+57.14%)
Mutual labels:  hive, etl, oracle, sqlserver
Datax
DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server
Stars: ✭ 116 (+452.38%)
Mutual labels:  hive, etl, oracle, sqlserver
Bigdata
💎🔥大数据学习笔记
Stars: ✭ 488 (+2223.81%)
Mutual labels:  hive, hbase, hdfs
BigDataTools
tools for bigData
Stars: ✭ 36 (+71.43%)
Mutual labels:  hive, hbase, hdfs
data-profiling
a set of scripts to pull meta data and data profiling metrics from relational database systems
Stars: ✭ 57 (+171.43%)
Mutual labels:  hive, oracle, sqlserver
Wifi
基于wifi抓取信息的大数据查询分析系统
Stars: ✭ 93 (+342.86%)
Mutual labels:  hive, hbase, hdfs
aaocp
一个对用户行为日志进行分析的大数据项目
Stars: ✭ 53 (+152.38%)
Mutual labels:  hive, hbase, hdfs
Repository
个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。
Stars: ✭ 92 (+338.1%)
Mutual labels:  hive, hbase, hdfs
Wedatasphere
WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!
Stars: ✭ 372 (+1671.43%)
Mutual labels:  hive, etl, hbase
Datafaker
Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具
Stars: ✭ 327 (+1457.14%)
Mutual labels:  hive, oracle, hbase
God Of Bigdata
专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+28509.52%)
Mutual labels:  hive, hbase, hdfs
Csv2db
The CSV to database command line loader
Stars: ✭ 102 (+385.71%)
Mutual labels:  etl, oracle, sqlserver
Bigdata Notes
大数据入门指南 ⭐
Stars: ✭ 10,991 (+52238.1%)
Mutual labels:  hive, hbase, hdfs
Bigdata docker
Big Data Ecosystem Docker
Stars: ✭ 161 (+666.67%)
Mutual labels:  hive, hbase, hdfs
Bigdataguide
大数据学习,从零开始学习大数据,包含大数据学习各阶段学习视频、面试资料
Stars: ✭ 817 (+3790.48%)
Mutual labels:  hive, hbase
Szt Bigdata
深圳地铁大数据客流分析系统🚇🚄🌟
Stars: ✭ 826 (+3833.33%)
Mutual labels:  hive, hbase
hive-metastore-client
A client for connecting and running DDLs on hive metastore.
Stars: ✭ 37 (+76.19%)
Mutual labels:  hive, etl
Docs4dev
后端开发常用框架文档及中文翻译,包含 Spring 系列文档(Spring, Spring Boot, Spring Cloud, Spring Security, Spring Session),大数据(Apache Hive, HBase, Apache Flume),日志(Log4j2, Logback),Http Server(NGINX,Apache),Python,数据库(OpenTSDB,MySQL,PostgreSQL)等最新官方文档以及对应的中文翻译。
Stars: ✭ 974 (+4538.1%)
Mutual labels:  hive, hbase
Luigi Warehouse
A luigi powered analytics / warehouse stack
Stars: ✭ 72 (+242.86%)
Mutual labels:  hive, etl

Datax-logo

DataX

DataX 是异构数据广泛使用的离线数据同步工具/平台,实现包括 MySQL、Oracle、SqlServer、Postgre、HDFS、Hive、ADS、HBase、OTS、ODPS 等各种异构数据源之间高效的数据同步功能。

Features

DataX本身作为数据同步框架,将不同数据源的同步抽象为从源头数据源读取数据的Reader插件,以及向目标端写入数据的Writer插件,理论上DataX框架可以支持任意数据源类型的数据同步工作。同时DataX插件体系作为一套生态系统, 每接入一套新数据源该新加入的数据源即可实现和现有的数据源互通。

DataX-OpenSource-DingTalk

Quick Start

Download Datax-bin下载地址
请点击:Quick Start

Support Data Channels

DataX目前已经有了比较全面的插件体系,主流的RDBMS数据库、NOSQL、大数据计算系统都已经接入,目前支持数据如下图,配置详情请点击图下列出DataX数据源参考指南

类型 数据源 Reader(读) Writer(写)
RDBMS 关系型数据库 Mysql
Oracle
SqlServer
Postgresql
达梦
阿里云数仓数据存储 ODPS
ADS
OSS
OCS
NoSQL数据存储 OTS
Hbase0.94
Hbase1.1
MongoDB
无结构化数据存储 TxtFile
FTP
HDFS

Support Data Channels List

Contact us

系统问题请及时提出在issue。请前往:DataxIssue

钉钉&微信扫描以下二维码加入交流群:

DataX-OpenSource-DingTalk

DataX-OpenSource-WeChat

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].