419 Open source projects that are alternatives of or similar to parquet-flinktacular

A collection of Apache Parquet add-on modules

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Flexible, Fast & Compact Serialization with RPC

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

GCS support for avro-tools, parquet-tools and protobuf

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Mu (μ) is a purely functional framework for building micro services.

A tool for data sampling, data generation, and data diffing

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Make record oriented data to columnar format.

Comparison and benchmark of JavaScript serialization libraries (Protocol Buffer, Avro, BSON, etc.)

Bucketing and partitioning system for Parquet

Iceberg is a table format for large, slow-moving tabular data

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

**Unofficial / Community** Kafka Connect MongoDB Sink Connector - Find the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

A fast Go Avro codec

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

A modular Protocol Buffers implementation for C

A collection of tools for extracting FHIR resources and analytics services on top of that data.

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

A CLI and Go client for Kafka Schema Registry

A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html

A collection of Magnolia add-on modules

Bazel rules for building Protobuf and gRPC code and libraries from proto_library targets

Record Query - A tool for doing record analysis and transformation

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Avro SerDe for Apache Spark structured APIs.

Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration

Scala library to eliminate boilerplate

CSVs sliced, diced & analyzed.

Confluent Schema Registry for Kafka

A micro server framework, support coroutine, and parallel-computing, used for building flatbuffers/thrift/protobuf/http protocol service.

Protocol Buffers implementation for J2CL

Kafka backed service library.

Avro support for kotlinx.serialization

Library to read a subset of Parquet files

Experiments with Apache Flink.

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.

Visual Studio Code integration for Buf.

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile

Ruby DSL to create Avro schemas

从本地IDEA提交Flink/Spark任务到Yarn/k8s集群

The metrics pipeline for DC/OS 1.9-1.11

Demo applications and code examples for Confluent Platform and Apache Kafka

Command Line Tool for managing Apache Kafka

A library provides consumer/producer to work with kafka, avro and schema registry

Haskell protocol-buffers package

A Thrift Parser built in TypeScript that generates a TypeScript AST that retains the Thrift grammar

Avrocado is a convenience library to handle Avro in Golang

Avro for JavaScript ⚡️

☀️ Avro, Protobuf, Thrift on Swagger

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration files, JSON protocol declaration files, and Avro IDL files.

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].