517 Open source projects that are alternatives of or similar to Cluster Pack

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Function, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

Python interface to the TileDB storage manager

JobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.

The Universal Storage Engine

Utils for streaming large files (S3, HDFS, gzip, bz2...)

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

Fundamentals of Spark with Python (using PySpark), code examples

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

Web tool for Kafka Connect |

⛈️ Rumble 1.11.0 "Banyan Tree"🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more

Kafka Connect FileSystem Connector

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

VGA and BIOS rom font extraction

Backtest 1000s of minute-by-minute trading algorithms for training AI with automated pricing data from: IEX, Tradier and FinViz. Datasets and trading performance automatically published to S3 for building AI training datasets for teaching DNNs how to trade. Runs on Kubernetes and docker-compose. >150 million trading history rows generated from +5000 algorithms. Heads up: Yahoo's Finance API was disabled on 2019-01-03 https://developer.yahoo.com/yql/

CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.

This is a repo documenting the best practices in PySpark.

Reliable PostgreSQL Backup & Restore

FUSE-based file system backed by Amazon S3

A fake S3 server written in NodeJs

A Python library that allows you to interact with Amazon S3 Buckets as if they are your local filesystem.

Parallel S3 and local filesystem execution tool.

gulp plugin to publish files to amazon s3

The Infinit policy-based software-defined storage platform.

AWS Toolkit for Visual Studio Code, an extension for working with AWS services including AWS Lambda.

Carthage cache for S3, Minio, Ceph, Google Storage, Artifactory and many others

Easy full stack backup operations on UNIX-like systems.

Official s3cmd repo -- Command line tool for managing Amazon S3 and CloudFront services

🦄 A modern web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...

Example project implementing best practices for PySpark ETL jobs and applications.

专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

🔑Auth and 📦Storage for Hasura. The quickest way to get Auth and Storage working for your next app based on Hasura.

Storage and image processing server written in Go

A web based file manager,web IDE / browser based code editor

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

Kafka Connect HDFS connector

Directly upload files to S3 compatible services with Django.

Modularized AWS SDK for JavaScript.

🚎 Notebook sharing hub

Helm plugin that allows to set up a chart repository in AWS S3.

Measure Amazon S3's performance from any location.

Tool for easy ClickHouse backup and restore with cloud storages support

EdgeFS - decentralized, scalable data fabric platform for Edge/IoT Computing and Kubernetes apps

Real Time Analytics and Data Pipelines based on Spark Streaming

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis.

NoDB isn't a database.. but it sort of looks like one.

✍️ Java 工程师必备架构体系知识总结：涵盖分布式、微服务、RPC等互联网公司常用架构，以及数据存储、缓存、搜索等必备技能

a high-performance, POSIX-ish Amazon S3 file system written in Go

Scalable, redundant, and distributed object store for Apache Hadoop

A collection of bash shell scripts for automating various tasks with Amazon Web Services using the AWS CLI and jq.

A simple mock implementation of the AWS S3 API startable as Docker image, JUnit 4 rule, or JUnit Jupiter extension

A boilerplate for writing PySpark Jobs

WinSCP is a popular free SFTP and FTP client for Windows, a powerful file manager that will improve your productivity. It supports also Amazon S3, FTPS, SCP and WebDAV protocols. Power users can automate WinSCP using .NET assembly.

High Performance, Kubernetes Native Object Storage

🔄 GitHub Action to sync a directory with a remote S3 bucket 🧺

Continuous Archiving for Postgres

Spark Gotchas. A subjective compilation of the Apache Spark tips and tricks

A library that allows you to easily mock out tests based on AWS infrastructure.

A vue-cli plugin that uploads your built Vue.js project to an S3 bucket

Amazon Simple Storage Service (S3) API Client