confluentinc / Kafka Connect Hdfs
Licence: other
Kafka Connect HDFS connector
Stars: ✭ 400
Programming Languages
java
68154 projects - #9 most used programming language
Projects that are alternatives of or similar to Kafka Connect Hdfs
kafka-connect-fs
Kafka Connect FileSystem Connector
Stars: ✭ 107 (-73.25%)
Mutual labels: hadoop, confluent, hdfs, apache-kafka
Bigdata Interview
🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
Stars: ✭ 857 (+114.25%)
Mutual labels: kafka, hadoop, hdfs
Sparta
Real Time Analytics and Data Pipelines based on Spark Streaming
Stars: ✭ 513 (+28.25%)
Mutual labels: kafka, hdfs, streaming
Kafka Connect Jdbc
Kafka Connect connector for JDBC-compatible databases
Stars: ✭ 698 (+74.5%)
Mutual labels: kafka, confluent, streaming
Hadoop For Geoevent
ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.
Stars: ✭ 5 (-98.75%)
Mutual labels: big-data, hadoop, hdfs
Streamx
kafka-connect-s3 : Ingest data from Kafka to Object Stores(s3)
Stars: ✭ 96 (-76%)
Mutual labels: kafka, big-data, streaming
bigdata-fun
A complete (distributed) BigData stack, running in containers
Stars: ✭ 14 (-96.5%)
Mutual labels: big-data, hadoop, hdfs
Bigdata Playground
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Stars: ✭ 177 (-55.75%)
Mutual labels: kafka, big-data, hadoop
God Of Bigdata
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
Stars: ✭ 6,008 (+1402%)
Mutual labels: kafka, hadoop, hdfs
Spark With Python
Fundamentals of Spark with Python (using PySpark), code examples
Stars: ✭ 150 (-62.5%)
Mutual labels: big-data, hadoop, hdfs
Hdfs Shell
HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS
Stars: ✭ 117 (-70.75%)
Mutual labels: big-data, hadoop, hdfs
Data Accelerator
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Stars: ✭ 247 (-38.25%)
Mutual labels: kafka, big-data, streaming
Kafka Ui
Open-Source Web GUI for Apache Kafka Management
Stars: ✭ 230 (-42.5%)
Mutual labels: apache-kafka, kafka, big-data
leaflet heatmap
简单的可视化湖州通话数据 假设数据量很大，没法用浏览器直接绘制热力图，把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后，再使用Apache Spark绘制热力图，然后用leafletjs加载OpenStreetMap图层和热力图图层，以达到良好的交互效果。现在使用Apache Spark实现绘制，可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法，并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .
Stars: ✭ 13 (-96.75%)
Mutual labels: big-data, hadoop, hdfs
Cp Demo
Confluent Platform Demo including Apache Kafka, ksqlDB, Control Center, Replicator, Confluent Schema Registry, Security
Stars: ✭ 278 (-30.5%)
Mutual labels: kafka, confluent
Kafka Connect HDFS Connector
kafka-connect-hdfs is a Kafka Connector for copying data between Kafka and Hadoop HDFS.
Documentation for this connector can be found here.
Development
To build a development version you'll need a recent version of Kafka as well as a set of upstream Confluent projects, which you'll have to build from their appropriate snapshot branch. See the FAQ for guidance on this process.
You can build kafka-connect-hdfs with Maven using the standard lifecycle phases.
FAQ
Refer frequently asked questions on Kafka Connect HDFS here - https://github.com/confluentinc/kafka-connect-hdfs/wiki/FAQ
Contribute
- Source Code: https://github.com/confluentinc/kafka-connect-hdfs
- Issue Tracker: https://github.com/confluentinc/kafka-connect-hdfs/issues
License
This project is licensed under the Confluent Community License.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].