138 Open source projects that are alternatives of or similar to parquet-extra

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Iceberg is a table format for large, slow-moving tabular data

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

How to use Parquet in Flink

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

GCS support for avro-tools, parquet-tools and protobuf

Make record oriented data to columnar format.

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

A tool for data sampling, data generation, and data diffing

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

A collection of Magnolia add-on modules

Kafka backed service library.

Avro support for kotlinx.serialization

A bank simulation application using mainly Clojure, which can be used to end-to-end test and show some graphs.

Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration

MongoDB Kafka Connector

Ruby DSL to create Avro schemas

The metrics pipeline for DC/OS 1.9-1.11

☀️ A tool for validating data using JSON Schema and converting JSON Schema documents into different data-interchange formats

A library provides consumer/producer to work with kafka, avro and schema registry

Avro for JavaScript ⚡️

Library to read a subset of Parquet files

A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html

**Unofficial / Community** Kafka Connect MongoDB Sink Connector - Find the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

Code examples that show to integrate Apache Kafka 0.8+ with Apache Storm 0.9+ and Apache Spark Streaming 1.1+, while using Apache Avro as the data serialization format.

Command Line Tool for managing Apache Kafka

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

Avro schema generation and serialization / deserialization for Scala

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration files, JSON protocol declaration files, and Avro IDL files.

CSVs sliced, diced & analyzed.

A fast Go Avro codec

Benchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++

Apache Avro is a data serialization system.

Demo applications and code examples for Confluent Platform and Apache Kafka

A collection of tools for extracting FHIR resources and analytics services on top of that data.

Avrocado is a convenience library to handle Avro in Golang

Flexible, Fast & Compact Serialization with RPC

☀️ Avro, Protobuf, Thrift on Swagger

Avro SerDe for Apache Spark structured APIs.

pmacct is a small set of multi-purpose passive network monitoring tools [NetFlow IPFIX sFlow libpcap BGP BMP RPKI IGP Streaming Telemetry].

Record Query - A tool for doing record analysis and transformation

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.

Web tool for Avro Schema Registry |

Gradle plugin to interact with Confluent Schema-Registry.

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

Showcases how to build a small Event-sourced application using Spring Boot, Spring Kafka, Apache Avro and Apache Kafka

Divolte Collector

Scala library to eliminate boilerplate

A SQL-like language for performing ETL transformations.

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile

Spark UDFs to deserialize Avro messages with schemas stored in Schema Registry.

Confluent Schema Registry for Kafka

Writing an Avro file is not as fast as you might want it. This is a library to write considerably faster to an avro file.

Generation of arbitrary case classes / ADTs instances with Scalaprops and Magnolia

Demonstrates how to content manage React apps with Magnolia CMS