517 Open source projects that are alternatives of or similar to Cluster Pack

Apache Spark for data engineers

🚚 Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

NGINX S3 Caching Gateway

Open Source Contributor Index

S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)

Upload manager using Laravel's built-in Filesystem/Cloud Storage

Minikube for big data with Scala and Spark

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs

⚡️ An efficient cache for the execution of dask graphs.

ArcGIS GeoEvent Server sample Hadoop connector for storing GeoEvents in HDFS.

This AWS SNS client library allows to publish messages to SNS that exceed the 256 KB message size limit.

Go io/fs.FS filesystem implementations for various URL schemes

Streaming replication for SQLite.

Go package for abstracting local, in-memory, and remote (Google Cloud Storage/S3) filesystems

Easy multipart uploads for Amazon S3, DigitalOcean Spaces and S3-compatible services

Simple and Distributed Machine Learning

Really fast sync tool for S3

Kafka Connect HDFS connector

Automated software delivery as fast and easy as possible 🚀

Terraform provider for managing minio S3 buckets and IAM Users

💾 Painless automated backups to multiple storage providers with Docker and duplicacy.

Automatic backups your web projects bases and files to the clouds via WebDAV.

Abstract file storage and upload plugin for CakePHP. Write to local disk, FTP, S3, Dropbox and more through a single interface. It's not just yet another uploader but a complete storage solution.

tools for bigData

🔥 Fast, Secure and Reliable System Backup, Set up in Minutes.

Directly upload files to S3 compatible services with Django.

ChubaoFS (abbrev. CBFS) is a cloud native distributed file system and object store.

Simple rails plugin that makes it easy to store uploaded files on Amazon S3

Evaluate the security of S3 buckets

Generic S3 server implementation

Rust library for interfacing with AWS S3 and other API compatible services

Data abstraction, storage, discovery, and serving system

Ruby server for tus resumable upload protocol

gulp plugin to publish files to amazon s3

Cloud Explorer

crux.land is a free registry service meant for hosting small (≤ 20kB) single deno scripts.

Amazon S3 input and output plugin for Fluentd

Distributed least squares approximation (dlsa) implemented with Apache Spark

Local Hashicorp Stack for DevOps Development without Hypervisor or Cloud

High throughput data ingestion logger to Fluentd, AWS S3 and Treasure Data

Terraform module to easily provision CloudFront CDN backed by an S3 origin

Collection of Terraform modules for frontend app deployment on AWS.

Modularized AWS SDK for JavaScript.

[DEPRECATED] A theoretically-infinitely-scalable Skyblock plugin.

A community AWS SDK for Perl Programmers

Django + S3

📫 A serverless email server on AWS using S3 and SES

A Terraform module that configures an s3 bucket for use with Terraform's remote state feature

MinIO Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage.

Helm plugin that allows to set up a chart repository in AWS S3.

Bulk Stash is a docker rclone service to sync, or copy, files between different storage services. For example, you can copy files either to or from a remote storage services like Amazon S3 to Google Cloud Storage, or locally from your laptop to a remote storage.

Machine Learning Course @ Santa Clara University

A simple cloud drive mapping web app supports local, FTP/SFTP, S3, OneDrive, WebDAV, Google Drive.

A simple guide to understand Prefect and make it work with your own docker-compose configuration.

Bucket Flaws ( S3 Bucket Mass Scanner ): A Simple Lightweight Script to Check for Common S3 Bucket Misconfigurations

Check read, write permissions on S3 buckets in your account

A pure python HDFS client

An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.

A React Starter App that displays how web developers can integrate their front end with AWS on the backend. The App interacts with AWS Cognito, API Gateway, Lambda and DynamoDB on the backend.