All Projects → bnjjj → chicon-rs

bnjjj / chicon-rs

Licence: MIT license
A file abstraction system for Rust

Programming Languages

rust
11053 projects

Projects that are alternatives of or similar to chicon-rs

acid-store
A library for secure, deduplicated, transactional, and verifiable data storage
Stars: ✭ 48 (-12.73%)
Mutual labels:  filesystem, s3, sftp
Afs
Abstract File Storage
Stars: ✭ 126 (+129.09%)
Mutual labels:  filesystem, s3
File Storage
File storage abstraction for Yii2
Stars: ✭ 116 (+110.91%)
Mutual labels:  filesystem, sftp
sftp-gateway
This repository contains a docker image configured to use the SSH File Transfer Protocol (SFTP) to transfer all its files to Cloud Blob Storage Services. This image can be deployed on a Kubernetes cluster with Helm.
Stars: ✭ 18 (-67.27%)
Mutual labels:  s3, sftp
S5cmd
Parallel S3 and local filesystem execution tool.
Stars: ✭ 565 (+927.27%)
Mutual labels:  filesystem, s3
S3fs Fuse
FUSE-based file system backed by Amazon S3
Stars: ✭ 5,733 (+10323.64%)
Mutual labels:  filesystem, s3
Tus Ruby Server
Ruby server for tus resumable upload protocol
Stars: ✭ 172 (+212.73%)
Mutual labels:  filesystem, s3
Flydrive
☁️ Flexible and Fluent framework-agnostic driver based system to manage storage in Node.js
Stars: ✭ 275 (+400%)
Mutual labels:  filesystem, s3
Vscode Remote Workspace
Multi protocol support for handling remote files like local ones in Visual Studio Code.
Stars: ✭ 197 (+258.18%)
Mutual labels:  filesystem, sftp
kafka-connect-fs
Kafka Connect FileSystem Connector
Stars: ✭ 107 (+94.55%)
Mutual labels:  filesystem, s3
nestjs-storage
Nestjs file system / file storage module wrapping flydrive
Stars: ✭ 92 (+67.27%)
Mutual labels:  filesystem, s3
Infinit
The Infinit policy-based software-defined storage platform.
Stars: ✭ 363 (+560%)
Mutual labels:  filesystem, s3
Goofys
a high-performance, POSIX-ish Amazon S3 file system written in Go
Stars: ✭ 3,932 (+7049.09%)
Mutual labels:  filesystem, s3
S3fs
Amazon S3 filesystem for PyFilesystem2
Stars: ✭ 111 (+101.82%)
Mutual labels:  filesystem, s3
Juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Stars: ✭ 4,262 (+7649.09%)
Mutual labels:  filesystem, s3
Mc
MinIO Client is a replacement for ls, cp, mkdir, diff and rsync commands for filesystems and object storage.
Stars: ✭ 1,962 (+3467.27%)
Mutual labels:  filesystem, s3
terraform-aws-sftp
This terraform module is used to create sftp on AWS for S3.
Stars: ✭ 20 (-63.64%)
Mutual labels:  s3, sftp
storage
Go package for abstracting local, in-memory, and remote (Google Cloud Storage/S3) filesystems
Stars: ✭ 49 (-10.91%)
Mutual labels:  filesystem, s3
Shrine
File Attachment toolkit for Ruby applications
Stars: ✭ 2,903 (+5178.18%)
Mutual labels:  filesystem, s3
Chubaofs
ChubaoFS (abbrev. CBFS) is a cloud native distributed file system and object store.
Stars: ✭ 2,482 (+4412.73%)
Mutual labels:  filesystem, s3

Chicon

Version Documentation

A file abstraction system for Rust. Chicon is a library meant to provide a simple, uniform and universal API interacting with any filesystem. Chicon behaves as an abstraction layer providing traits, types and methods. The main FileSystem trait is based on the usage of std::fs::* in order to be transparent when switching from a physical filesystem to a virtual one like S3, SFTP, SSH or in-memory. Chicon is suitable for any situation where storing directories and files on different filesystems is needed. Memory file system can be appropriate to write tests in order to have a faster execution than with an IO filesystem.

Examples

Use S3 as backend to create a file

use std::io::prelude::*;
use chicon::{DirEntry, File, FileSystem, S3FileSystem};
let s3_fs = S3FileSystem::new(
     String::from("my_access_key_id"),
     String::from("secret_access_key"),
     String::from("my_bucket"),
     String::from("my_region"),
     String::from("http://127.0.0.1"), // endpoint
);
let mut file = s3_fs.create_file("test.test").unwrap()
file.write_all(String::from("here is a test").as_bytes()).unwrap();
file.sync_all().unwrap();
let mut content: String = String::new();
file.read_to_string(&mut content).unwrap();
assert_eq!(content, String::from("here is a test"));
s3_fs.remove_file("test.test").unwrap(); // To delete the file

Use SFTP as backend to create a file

You just need to change from S3FileSystem::new to SFTPFileSystem::new.

use std::io::prelude::*;
use chicon::{DirEntry, File, FileSystem, SFTPFileSystem};
let sftp_fs = SFTPFileSystem::new(
    String::from("127.0.0.1:2222"), // host:port
    String::from("foo"), // user
    None, // Some("passphrase") if you have a passphrase configured on your ssh key
    "/Users/foo/.ssh/my_private_key", // ABSOLUTE path to private key
    "/Users/foo/.ssh/my_public_key.pub" // ABSOLUTE path to public key
);
let mut file = sftp_fs.create_file("test.test").unwrap()
file.write_all(String::from("here is a test").as_bytes()).unwrap();
file.sync_all().unwrap();
let mut content: String = String::new();
file.read_to_string(&mut content).unwrap();
assert_eq!(content, String::from("here is a test"));

Use SSH as backend to read a file

use std::io::prelude::*;
use chicon::{DirEntry, File, FileSystem, SSHFileSystem};
let ssh_fs = SSHFileSystem::new(
    String::from("127.0.0.1:2222"), // host:port
    String::from("foo"), // user
    None, // Some("passphrase") if you have a passphrase configured on your ssh key
    "/Users/foo/.ssh/my_private_key", // ABSOLUTE path to private key
    "/Users/foo/.ssh/my_public_key.pub" // ABSOLUTE path to public key
);
let mut file = ssh_fs.open_file("share/myfile.txt").unwrap();
let mut buffer = String::new();
file.read_to_string(&mut buffer).unwrap();
println!("Here is the content of your file: {}", buffer);

Use OS (local filesystem) as backend to create and read a directory

use std::io::prelude::*;
use chicon::{DirEntry, File, FileType, FileSystem, OsFileSystem};
let os_fs = OsFileSystem::new();
os_fs.create_dir_all("testreaddir/test").unwrap();
os_fs.create_file("testreaddir/mytest.test").unwrap();
let dir_entries = os_fs.read_dir("testreaddir").unwrap();
assert!(!dir_entries.is_empty())
assert_eq!(dir_entries.len(), 2)
assert_eq!(
    dir_entries.get(0).unwrap().path().unwrap(),
    PathBuf::from("testreaddir/test")
);
assert_eq!(
    dir_entries.get(0).unwrap().file_type().unwrap(),
    FileType::Directory
);
std::fs::remove_dir_all("testreaddir").unwrap(); // To remove dir and all entries inside

If you need more examples, checkout the tests in the source code on the GitHub repository.

Roadmap

  • implement swift as a new backend
  • refactor with more idiomatic Rust stuff
  • add async support
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].