ChoetlETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Stars: ✭ 372 (+1182.76%)
Mutual labels: avro, parquet
Gcs ToolsGCS support for avro-tools, parquet-tools and protobuf
Stars: ✭ 57 (+96.55%)
Mutual labels: avro, parquet
IcebergIceberg is a table format for large, slow-moving tabular data
Stars: ✭ 393 (+1255.17%)
Mutual labels: avro, parquet
DaFlowApache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.
Stars: ✭ 24 (-17.24%)
Mutual labels: avro, parquet
NoprotoFlexible, Fast & Compact Serialization with RPC
Stars: ✭ 138 (+375.86%)
Mutual labels: avro, protocol-buffers
javascript-serialization-benchmarkComparison and benchmark of JavaScript serialization libraries (Protocol Buffer, Avro, BSON, etc.)
Stars: ✭ 54 (+86.21%)
Mutual labels: avro, protocol-buffers
Cpp SerializersBenchmark comparing various data serialization libraries (thrift, protobuf etc.) for C++
Stars: ✭ 533 (+1737.93%)
Mutual labels: avro, thrift
Devops Python Tools80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
Stars: ✭ 406 (+1300%)
Mutual labels: avro, parquet
SchemerSchema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.
Stars: ✭ 97 (+234.48%)
Mutual labels: avro, parquet
Bigdata File ViewerA cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.
Stars: ✭ 86 (+196.55%)
Mutual labels: avro, parquet
columnifyMake record oriented data to columnar format.
Stars: ✭ 28 (-3.45%)
Mutual labels: avro, parquet
Mu HaskellMu (μ) is a purely functional framework for building micro services.
Stars: ✭ 215 (+641.38%)
Mutual labels: avro, protocol-buffers
parquet-extraA collection of Apache Parquet add-on modules
Stars: ✭ 30 (+3.45%)
Mutual labels: avro, parquet
RatatoolA tool for data sampling, data generation, and data diffing
Stars: ✭ 279 (+862.07%)
Mutual labels: avro, parquet
PucketBucketing and partitioning system for Parquet
Stars: ✭ 29 (+0%)
Mutual labels: thrift, parquet
Rumble⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more
Stars: ✭ 58 (+100%)
Mutual labels: avro, parquet
Bigdata PlaygroundA complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
Stars: ✭ 177 (+510.34%)
Mutual labels: avro, parquet
Vscode Data PreviewData Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files
Stars: ✭ 245 (+744.83%)
Mutual labels: avro, parquet
rules proto grpcBazel rules for building Protobuf and gRPC code and libraries from proto_library targets
Stars: ✭ 201 (+593.1%)
Mutual labels: protocol-buffers
kafka-scala-examplesExamples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala
Stars: ✭ 53 (+82.76%)
Mutual labels: avro