All Projects → artefactory → sftp-gateway

artefactory / sftp-gateway

Licence: LGPL-3.0 license
This repository contains a docker image configured to use the SSH File Transfer Protocol (SFTP) to transfer all its files to Cloud Blob Storage Services. This image can be deployed on a Kubernetes cluster with Helm.

Programming Languages

python
139335 projects - #7 most used programming language
Makefile
30231 projects
shell
77523 projects
Mustache
554 projects
Dockerfile
14818 projects

Projects that are alternatives of or similar to sftp-gateway

Winscp
WinSCP is a popular free SFTP and FTP client for Windows, a powerful file manager that will improve your productivity. It supports also Amazon S3, FTPS, SCP and WebDAV protocols. Power users can automate WinSCP using .NET assembly.
Stars: ✭ 794 (+4311.11%)
Mutual labels:  s3, sftp
Drone Cache
A Drone plugin for caching current workspace files between builds to reduce your build times
Stars: ✭ 194 (+977.78%)
Mutual labels:  s3, sftp
Cyberduck
Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.
Stars: ✭ 1,080 (+5900%)
Mutual labels:  s3, sftp
Goofys
a high-performance, POSIX-ish Amazon S3 file system written in Go
Stars: ✭ 3,932 (+21744.44%)
Mutual labels:  s3, gcs
hub
Public reusable components for Polyaxon
Stars: ✭ 8 (-55.56%)
Mutual labels:  s3, gcs
Filestash
🦄 A modern web client for SFTP, S3, FTP, WebDAV, Git, Minio, LDAP, CalDAV, CardDAV, Mysql, Backblaze, ...
Stars: ✭ 5,231 (+28961.11%)
Mutual labels:  s3, sftp
S3 Sftp Proxy
An AWS S3 gateway proxying SFTP connections.
Stars: ✭ 112 (+522.22%)
Mutual labels:  s3, sftp
storage
Go library providing common interface for working across multiple cloud storage backends
Stars: ✭ 154 (+755.56%)
Mutual labels:  s3, gcs
Stock Analysis Engine
Backtest 1000s of minute-by-minute trading algorithms for training AI with automated pricing data from: IEX, Tradier and FinViz. Datasets and trading performance automatically published to S3 for building AI training datasets for teaching DNNs how to trade. Runs on Kubernetes and docker-compose. >150 million trading history rows generated from +5000 algorithms. Heads up: Yahoo's Finance API was disabled on 2019-01-03 https://developer.yahoo.com/yql/
Stars: ✭ 605 (+3261.11%)
Mutual labels:  helm, s3
Helm S3
Helm plugin that allows to set up a chart repository in AWS S3.
Stars: ✭ 372 (+1966.67%)
Mutual labels:  helm, s3
docker base images
Vlad's Base Images for Docker
Stars: ✭ 61 (+238.89%)
Mutual labels:  s3, inotify
terraform-aws-sftp
This terraform module is used to create sftp on AWS for S3.
Stars: ✭ 20 (+11.11%)
Mutual labels:  s3, sftp
astro
Astro allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Stars: ✭ 79 (+338.89%)
Mutual labels:  s3, gcs
Rclone
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Yandex Files
Stars: ✭ 30,541 (+169572.22%)
Mutual labels:  s3, sftp
acid-store
A library for secure, deduplicated, transactional, and verifiable data storage
Stars: ✭ 48 (+166.67%)
Mutual labels:  s3, sftp
Iostreams
IOStreams is an incredibly powerful streaming library that makes changes to file formats, compression, encryption, or storage mechanism transparent to the application.
Stars: ✭ 84 (+366.67%)
Mutual labels:  s3, sftp
go-fsimpl
Go io/fs.FS filesystem implementations for various URL schemes
Stars: ✭ 225 (+1150%)
Mutual labels:  s3, gcs
chicon-rs
A file abstraction system for Rust
Stars: ✭ 55 (+205.56%)
Mutual labels:  s3, sftp
Sftpgo
Fully featured and highly configurable SFTP server with optional HTTP, FTP/S and WebDAV support - S3, Google Cloud Storage, Azure Blob
Stars: ✭ 3,534 (+19533.33%)
Mutual labels:  s3, sftp
docker-sftp
SFTP Server for Docker
Stars: ✭ 118 (+555.56%)
Mutual labels:  helm, sftp

sftp-gateway

Architecture

Nautilus SFTP Gateway architecture

A Docker image containing a SSH server and an INotify daemon allows to create a SFTP gateway server. Once deployed, you can connect to the SFTP server and read/write files that are immediately synchronised to GCS, S3, Azure Blob Storage (in development) and/or other cloud storage services.

The repository contains the appropriate files to deploy the container to Kubernetes.

Maintainer

Overview

When you run a container based on this image, it creates a SFTP server that can only be accessed by specific users, and moves to uploaded data to one or more specific buckets.

The users and the buckets are provided at runtime via container Environment variables. When the container starts, it uses the Environment variables to generate the appropriate configuration files and start the services. The container does not persist any data by default.

The docker image does not contain any credentials, they must be provided during the container deployment via a mounted secrets volume on Kubernetes, or a mounted volume for vanilla Docker. See below for more information.

Quick setup

  • 1/ Setup your environment by running export ENV=dev (here dev for example). Also run the following command : export PYTHONPATH=$PYTHONPATH:.
  • 2/ Fill in your configuration by copying the ./config/example.yaml into a new ./config/${ENV}.yaml file and replace the values by the ones you need for your project.
  • 3/ Add your secrets into the ./credentials/${ENV} folder or let the project generate them for you.
  • 4/ Test your configuration by running the following command : make docker_run
  • 5/ Generate or copy an existing external IPv4 address that can be used by your Kubernetes cluster in your configuration file under APP -> SERVICE_IP.
  • 6/ Deploy the SFTP on Kubernetes by running the following command : make helm_install

Table of contents

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].