154 Open source projects that are alternatives of or similar to Petastorm

Updated repository

BERT, AWS RDS, AWS Forecast, EMR Spark Cluster, Hive, Serverless, Google Assistant + Raspberry Pi, Infrared, Google Cloud Platform Natural Language, Anomaly detection, Tensorflow, Mathematics

A tool for building feature stores.

pyspark🍒🥭 is delicious，just eat it!😋😋

🐍 Quick reference guide to common patterns & functions in PySpark.

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs

Apache (Py)Spark type annotations (stub files).

Relation Extraction using Deep learning(CNN)

Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

PySpark Code for Hands-on Learners

[NOT MAINTAINED] Predicting Bit coin price using Time series analysis and sentiment analysis of tweets on bitcoin

Spark 2.0 Python Machine Learning examples

Word2Vec models with Twitter data using Spark. Blog:

🌐 Interactive Workshop on GeoAnalysis using PySpark

Data Preview 🈸 extension for importing 📤 viewing 🔎 slicing 🔪 dicing 🎲 charting 📊 & exporting 📥 large JSON array/config, YAML, Apache Arrow, Avro, Parquet & Excel data files

Manipulate arrays of complex data structures as easily as Numpy.

fully asynchronous, pure JavaScript implementation of the Parquet file format

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

A SQLite vtable extension to read Parquet files

Simple windows desktop application for viewing & querying Apache Parquet files

Apache Parquet implementation in Rust

A consistent table management library in python

Big Data Toolkit for the JVM

A large-scale entity and relation database supporting aggregation of properties

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Amazon S3 Find and Forget is a solution to handle data erasure requests from data lakes stored on Amazon S3, for example, pursuant to the European General Data Protection Regulation (GDPR)

Go package to read and write parquet files. parquet is a file format to store nested data structures in a flat columnar data format. It can be used in the Hadoop ecosystem and with tools such as Presto and AWS Athena.

Spark SQL index for Parquet tables

Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.

Schema registry for CSV, TSV, JSON, AVRO and Parquet schema. Supports schema inference and GraphQL API.

Apache Parquet

A cross-platform (Windows, MAC, Linux) desktop application to view common bigdata binary format like Parquet, ORC, AVRO, etc. Support local file system, HDFS, AWS S3, Azure Blob Storage ,etc.

Read SparkSQL parquet file as RDD[Protobuf]

Apache Drill is a distributed MPP query layer for self describing data