WirbelsturmWirbelsturm is a Vagrant and Puppet based tool to perform 1-click local and remote deployments, with a focus on big data tech like Kafka.
Stars: ✭ 332 (-54.4%)
Agile data code 2Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
Stars: ✭ 413 (-43.27%)
Awesome PulsarA curated list of Pulsar tools, integrations and resources.
Stars: ✭ 57 (-92.17%)
Azure Event Hubs SparkEnabling Continuous Data Processing with Apache Spark and Azure Event Hubs
Stars: ✭ 140 (-80.77%)
SlimmessagebusLightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.
Stars: ✭ 120 (-83.52%)
AbrisAvro SerDe for Apache Spark structured APIs.
Stars: ✭ 130 (-82.14%)
KafkactlCommand Line Tool for managing Apache Kafka
Stars: ✭ 177 (-75.69%)
OryxOryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
Stars: ✭ 1,785 (+145.19%)
Bigdata PlaygroundA complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Stars: ✭ 177 (-75.69%)
Storm Dynamic SpoutA framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
Stars: ✭ 40 (-94.51%)
Data AcceleratorData Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
Stars: ✭ 247 (-66.07%)
MeetupKafka 한국 사용자 모임에서 운영하는 meetup repository
Stars: ✭ 106 (-85.44%)
WaterdropWaterDrop is a standalone Karafka component library for generating Kafka messages
Stars: ✭ 136 (-81.32%)
Azkarra Streams🚀 Azkarra is a lightweight java framework to make it easy to develop, deploy and manage cloud-native streaming microservices based on Apache Kafka Streams.
Stars: ✭ 146 (-79.95%)
Pmacctpmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].
Stars: ✭ 677 (-7.01%)
FreestyleA cohesive & pragmatic framework of FP centric Scala libraries
Stars: ✭ 627 (-13.87%)
fink-brokerAstronomy Broker based on Apache Spark
Stars: ✭ 18 (-97.53%)
kafka-serializationLego bricks to build Apache Kafka serializers and deserializers
Stars: ✭ 122 (-83.24%)
KukulcanA REPL for Apache Kafka
Stars: ✭ 103 (-85.85%)
KarafkaFramework for Apache Kafka based Ruby and Rails applications development.
Stars: ✭ 1,223 (+67.99%)
Kattlo CliKattlo CLI Project
Stars: ✭ 58 (-92.03%)
KopKafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar
Stars: ✭ 159 (-78.16%)
DaFlowApache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (-96.7%)
kafka-compose🎼 Docker compose files for various kafka stacks
Stars: ✭ 32 (-95.6%)
autThe Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Stars: ✭ 111 (-84.75%)
LibrdkafkaThe Apache Kafka C/C++ library
Stars: ✭ 5,617 (+671.57%)
Kafka UiOpen-Source Web GUI for Apache Kafka Management
Stars: ✭ 230 (-68.41%)
Cp All In Onedocker-compose.yml files for cp-all-in-one , cp-all-in-one-community, cp-all-in-one-cloud
Stars: ✭ 239 (-67.17%)
InsulatorA client UI to inspect Kafka topics, consume, produce and much more
Stars: ✭ 53 (-92.72%)
leaflet heatmap简单的可视化湖州通话数据 假设数据量很大,没法用浏览器直接绘制热力图,把绘制热力图这一步骤放到线下计算分析。使用Apache Spark并行计算数据之后,再使用Apache Spark绘制热力图,然后用leafletjs加载OpenStreetMap图层和热力图图层,以达到良好的交互效果。现在使用Apache Spark实现绘制,可能是Apache Spark不擅长这方面的计算或者是我没有设计好算法,并行计算的速度比不上单机计算。Apache Spark绘制热力图和计算代码在这 https://github.com/yuanzhaokang/ParallelizeHeatmap.git .
Stars: ✭ 13 (-98.21%)
PykafkaApache Kafka client for Python; high-level & low-level consumer/producer, with great performance.
Stars: ✭ 1,079 (+48.21%)
Spark NotebookInteractive and Reactive Data Science using Scala and Spark.
Stars: ✭ 3,081 (+323.21%)
ZatZeek Analysis Tools (ZAT): Processing and analysis of Zeek network data with Pandas, scikit-learn, Kafka and Spark
Stars: ✭ 303 (-58.38%)
Spark Jupyter AwsA guide on how to set up Jupyter with Pyspark painlessly on AWS EC2 clusters, with S3 I/O support
Stars: ✭ 259 (-64.42%)
Learningsparkv2This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
Stars: ✭ 307 (-57.83%)
Coolplayspark酷玩 Spark: Spark 源代码解析、Spark 类库等
Stars: ✭ 3,318 (+355.77%)
KqKafka-based Job Queue for Python
Stars: ✭ 530 (-27.2%)
SparkmeasureThis is the development repository of SparkMeasure, a tool for performance troubleshooting of Apache Spark workloads. It simplifies the collection and analysis of Spark task metrics data.
Stars: ✭ 368 (-49.45%)
Kafka Sprout🚀 Web GUI for Kafka Cluster Management
Stars: ✭ 388 (-46.7%)
WedatasphereWeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!
Stars: ✭ 372 (-48.9%)
IcebergIceberg is a table format for large, slow-moving tabular data
Stars: ✭ 393 (-46.02%)
SpartaReal Time Analytics and Data Pipelines based on Spark Streaming
Stars: ✭ 513 (-29.53%)
Devops Python Tools80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Stars: ✭ 406 (-44.23%)
CppkafkaModern C++ Apache Kafka client library (wrapper for librdkafka)
Stars: ✭ 413 (-43.27%)