296 Open source projects that are alternatives of or similar to columnify

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

GCS support for avro-tools, parquet-tools and protobuf

ETL Framework for .NET / c# (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

A collection of Apache Parquet add-on modules

Apache Avro is a data serialization system.

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

A tool for data sampling, data generation, and data diffing

Kotlin Bigdata Toolkit

Iceberg is a table format for large, slow-moving tabular data

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

How to use Parquet in Flink

Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.

Golang Client for Schema Registry

CSVs sliced, diced & analyzed.

java学习和项目中一些典型的应用场景样例代码

Anovos - An Open Source Library for Scalable feature engineering Using Apache-Spark

SQL Parsers for BigData, built with antlr4.

Full featured TypeScript Twitter archive reader and browser

Plugin SBT to Generate Scala classes from Apache Avro schemas hosted on a remote Confluent Schema Registry.

Avro Serialisation/Deserialisation (SerDe) library for PHP 7.3+ & 8.0 with a Symfony Serializer integration

大数据框架 Spark MLlib 机器学习库基础算法全面讲解,附带齐全的测试文件

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

个人笔记集中营，快糙猛的形式记录技术性Notes .. 📚☕️⌨️🎧

Amas is recursive acronym for “Amas, monitor alert system”.

一道面试题的思考 - 6000万数据包和300万数据包在50M内存使用环境中求交集

大数据生态圈学习

Library to read a subset of Parquet files

Albis: High-Performance File Format for Big Data Systems

Examples of Avro, Kafka, Schema Registry, Kafka Streams, Interactive Queries, KSQL, Kafka Connect in Scala

This code creates a Kinesis Firehose in AWS to send CloudWatch log data to S3.

Hayabusa: Simple and Fast Full-Text Search Engine for Massive System Log Data

bigdata note

Avrow is a pure Rust implementation of the avro specification https://avro.apache.org/docs/current/spec.html with Serde support.

Standalone alternatives to Kafka Connect Connectors

Multi docker container images for main Big Data Tools. (Hadoop, Spark, Kafka, HBase, Cassandra, Zookeeper, Zeppelin, Drill, Flink, Hive, Hue, Mesos, ... )

Short course on interactive analysis of Big Earth Data with Jupyter Notebooks

An open-source, cloud-ready web application for simplified deployment of big data workflows.

A collection of tools for extracting FHIR resources and analytics services on top of that data.

GreyCat - Data Analytics, Temporal data, What-if, Live machine learning

Examples for gauravbytes.com

This is a learning note | Java基础，JVM，源码，大数据，面经

A PHP 7.3+ API client for the Confluent Schema Registry REST API based on Guzzle 6 - http://docs.confluent.io/current/schema-registry/docs/index.html

Масштабируемое машинное обучение и анализ больших данных с Apache Spark

bigdatatutorial

Measuring the performance of popular streaming engines with Yahoo's Streaming Benchmark

Writing an Avro file is not as fast as you might want it. This is a library to write considerably faster to an avro file.

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Uber-project for standard Jackson binary format backends: avro, cbor, ion, protobuf, smile

大数据学习笔记，学习路线，技术案例整理。

Mu (μ) is a purely functional framework for building micro services.

Scalable package delivery logistics simulator built using SingleStore and Vectorized Redpanda

All code samples, scripts and more in-depth examples for The Apache Ignite Book. Include Apache Ignite 2.6 or above

TiDB connectors for Flink/Hive/Presto

Command Line Tool for managing Apache Kafka

A Gradle plugin to allow easily performing Java code generation for Apache Avro. It supports JSON schema declaration files, JSON protocol declaration files, and Avro IDL files.

编程路上加油站！------【持续更新中...欢迎star,欢迎常回来看看......】【内容：编程/学习/阅读资源，开源项目,面试题,网站,书,博客,教程等等】