ob bulkstashBulk Stash is a docker rclone service to sync, or copy, files between different storage services. For example, you can copy files either to or from a remote storage services like Amazon S3 to Google Cloud Storage, or locally from your laptop to a remote storage.
Stars: ✭ 113 (+182.5%)
LuigiLuigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Stars: ✭ 15,226 (+37965%)
Rclone"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
Stars: ✭ 30,541 (+76252.5%)
aws-pdf-textract-pipeline🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Stars: ✭ 141 (+252.5%)
PyFiSyncPython (+ rsync or rclone) based intelligent file sync with automatic backups and file move/delete tracking.
Stars: ✭ 88 (+120%)
Docker S3 VolumeDocker container with a data volume from s3.
Stars: ✭ 166 (+315%)
FlinkxBased on Apache Flink. support data synchronization/integration and streaming SQL computation.
Stars: ✭ 2,651 (+6527.5%)
acid-storeA library for secure, deduplicated, transactional, and verifiable data storage
Stars: ✭ 48 (+20%)
mindavA self-hosted file backup server which bridges WebDAV protocol with @minio written in @totoval. Webdav ❤️ Minio
Stars: ✭ 64 (+60%)
AkubraSimple solution to keep a independent S3 storages in sync
Stars: ✭ 79 (+97.5%)
nifiDeploy a secured, clustered, auto-scaling NiFi service in AWS.
Stars: ✭ 37 (-7.5%)
Scrapy S3pipelineScrapy pipeline to store chunked items into Amazon S3 or Google Cloud Storage bucket.
Stars: ✭ 57 (+42.5%)
jobAnalytics and searchJobAnalytics system consumes data from multiple sources and provides valuable information to both job hunters and recruiters.
Stars: ✭ 25 (-37.5%)
re-moteRe-mote operations using SSH and Re-gent
Stars: ✭ 61 (+52.5%)
Drone CacheA Drone plugin for caching current workspace files between builds to reduce your build times
Stars: ✭ 194 (+385%)
minio-rclone-webdav-serverA @rclone served WebDAV server with @minio as the s3 storage backend docker example
Stars: ✭ 17 (-57.5%)
rclone-drive☁️Simple web cloud storage based on rclone, transform cloud storage (s3, google drive, one drive, dropbox) into own custom web-based storage
Stars: ✭ 30 (-25%)
S4🔄 Fast and cheap synchronisation of files using Amazon S3
Stars: ✭ 69 (+72.5%)
S3syncReally fast sync tool for S3
Stars: ✭ 224 (+460%)
datajobBuild and deploy a serverless data pipeline on AWS with no effort.
Stars: ✭ 101 (+152.5%)
myplanet🌕 myPlanet android app reads data from 🌎 for offline use as well as it collect usage data and sends them back to the Planet.
Stars: ✭ 17 (-57.5%)
diffidoWatch web pages for changes
Stars: ✭ 19 (-52.5%)
backbeatZenko Backbeat is the core engine for asynchronous replication, optimized for queuing metadata updates and dispatching work to long-running tasks in the background.
Stars: ✭ 51 (+27.5%)
grape-nfAn automated RNA-seq pipeline using Nextflow
Stars: ✭ 30 (-25%)
HTGSThe Hybrid Task Graph Scheduler API
Stars: ✭ 36 (-10%)
LTRpredDe novo annotation of young retrotransposons
Stars: ✭ 35 (-12.5%)
errgrouperrgroup with goroutine worker limits
Stars: ✭ 143 (+257.5%)
ckwinC-Kermit for Windows - scriptable internet and serial communications with terminal emulation
Stars: ✭ 35 (-12.5%)
ClarityNLPAn NLP framework for clinical phenotyping. Docker | Python | Solr | OMOP. http://claritynlp.readthedocs.io/en/latest/
Stars: ✭ 100 (+150%)
sync-forkthe quickest way to sync your fork: npx sync-fork
Stars: ✭ 15 (-62.5%)
aws-tailorAWS account provisioning and management service
Stars: ✭ 105 (+162.5%)
BACTpipeBACTpipe: An assembly and annotation pipeline for bacterial genomics
Stars: ✭ 19 (-52.5%)
skbnCopy files and directories between Kubernetes and cloud storage
Stars: ✭ 68 (+70%)
mongomeiliKeep your Mongoose Schemas synced with MeiliSearch
Stars: ✭ 33 (-17.5%)
terraform-aws-s3Terraform module to create default S3 bucket with logging and encryption type specific features.
Stars: ✭ 22 (-45%)
thingshub[UNMAINTAINED] Synchronize issues assigned to you from a Github repo into Things.
Stars: ✭ 17 (-57.5%)
Any-file-to-Google-DriveThis Google Colab notebook will help you download any file directly to Google Drive with the help of the JDownloader web interface
Stars: ✭ 47 (+17.5%)
local-cloudTurn any computer at home into a cloud for easy sharing of files across your devices.
Stars: ✭ 70 (+75%)
data-transfer-hubSeamless User Interface for replicating data into AWS.
Stars: ✭ 102 (+155%)
watHow fast are computers?
Stars: ✭ 26 (-35%)
xpA framework (comand line tool + libraries) for creating flexible compute pipelines
Stars: ✭ 54 (+35%)
docker-volume-rcloneUse Rclone as a backend for docker volume (also as a plugin). This permit to easely mount a lot of cloud provider (https://rclone.org/overview).
Stars: ✭ 97 (+142.5%)
adonis-bundlerBlazing fast, zero configuration assets bundler for AdonisJS
Stars: ✭ 19 (-52.5%)
pipelineRDA chain of responsability pattern implementation in .NET that supports retry policy, sync and async steps, rollback, pipeline recovery by cache and visual documentation using diagrams.
Stars: ✭ 19 (-52.5%)
CloudConductorCloudConductor is a workflow management system that generates and executes bioinformatics pipelines
Stars: ✭ 13 (-67.5%)
nestjs-storageNestjs file system / file storage module wrapping flydrive
Stars: ✭ 92 (+130%)
TabSyncA lightweight synchronizer between Android's Tabs and Lists. Available on the View system and Jetpack Compose.
Stars: ✭ 98 (+145%)
gawnGenome Annotation Without Nightmares
Stars: ✭ 35 (-12.5%)
commercetools-project-syncDockerized CLI application which allows to automatically sync different resources between commercetools projects
Stars: ✭ 26 (-35%)