282 Open source projects that are alternatives of or similar to Parquet Rs

A large-scale entity and relation database supporting aggregation of properties

Apache-Spark based Data Flow(ETL) Framework which supports multiple read, write destinations of different types and also support multiple categories of transformation rules.

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

The Hadoop ETL UDFs are the main way to load data from Hadoop into EXASOL

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

Iceberg is a table format for large, slow-moving tabular data

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

Big Data Toolkit for the JVM

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Apache Drill is a distributed MPP query layer for self describing data

DataX is an open source universal ETL tool that support Cassandra, ClickHouse, DBF, Hive, InfluxDB, Kudu, MySQL, Oracle, Presto(Trino), PostgreSQL, SQL Server

个人学习知识库涉及到数据仓库建模、实时计算、大数据、Java、算法等。

Python client for Hadoop® YARN API

Apache Parquet

Ephemeral Hadoop clusters using Google Compute Platform

Asakusa Framework

Cookbook to install Hadoop 2.0+ using Chef

Multiple node cluster on Docker for self development.

Mirror of Linkedin's Camus

Road to Machine Learning

Mirror of Apache Chukwa

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

A pandas-like deferred expression system, with first-class SQL support

基于wifi抓取信息的大数据查询分析系统

An Airflow docker image preconfigured to work well with Spark and Hadoop/EMR

Mirror of Apache Hadoop MapReduce

Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

HBase客户端数据管理软件

Read SparkSQL parquet file as RDD[Protobuf]

Support TensorFlow on YARN

Griffon Data Science Virtual Machine

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

🚢 Docker image for Apache Spark

extremely distributed machine learning

Train TensorFlow models on YARN in just a few lines of code!

Production Ready Data Integration Product, documentation：

Axibase Time Series Database Documentation

Apache Hadoop docker image

🐘 Source code for assignments of Udacity course "Introduction to Hadoop and MapReduce"

Hive UDFs for funnel analysis

Spark SQL index for Parquet tables

A light-weight distributed stream computing framework for Golang

Jumbune, an open source BigData APM & Data Quality Management Platform for Data Clouds. Enterprise feature offering is available at http://jumbune.com. More details of open source offering are at,

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Mirror of Apache Calcite - Avatica

A tool for scale and performance testing of HDFS with a specific focus on the NameNode.

80+ HAProxy Configs for Hadoop, Big Data, NoSQL, Docker, Elasticsearch, SolrCloud, HBase, MySQL, PostgreSQL, Apache Drill, Hive, Presto, Impala, Hue, ZooKeeper, SSH, RabbitMQ, Redis, Riak, Cloudera, OpenTSDB, InfluxDB, Prometheus, Kibana, Graphite, Rancher etc.

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

An implementation of locality sensitive hashing with Hadoop

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

GCS support for avro-tools, parquet-tools and protobuf

HDFS Shell is a HDFS manipulation tool to work with functions integrated in Hadoop DFS

大数据入门指南 ⭐