GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → radogost → ucz-dfs

radogost / ucz-dfs

Licence: Apache-2.0 license
A distributed file system written in Rust.

Programming Languages

rust
11053 projects

Labels

distributed-systemsfilesystemhdfsdistributed-file-system

Projects that are alternatives of or similar to ucz-dfs

Seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
Stars: ✭ 13,380 (+53420%)
Mutual labels:  distributed-systems, hdfs, distributed-file-system
Juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Stars: ✭ 4,262 (+16948%)
Mutual labels:  distributed-systems, filesystem, hdfs
file management sys
file_management_sys 是一个文件共享系统，包括前端文件展示系统和后台管理系统，基于SpringBoot + MyBatis实现。前端文件展示系统包括文件分类和展示界面，文件搜索和文件上传等模块。后台管理系统包含文件管理，权限管理等模块。
Stars: ✭ 60 (+140%)
Mutual labels:  distributed-systems, filesystem
Go Fastdfs
go-fastdfs 是一个简单的分布式文件系统(私有云存储)，具有无中心、高性能，高可靠，免维护等优点，支持断点续传，分块上传，小文件合并，自动同步，自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic r…
Stars: ✭ 2,923 (+11592%)
Mutual labels:  filesystem, distributed-file-system
Glusterfs
Gluster Filesystem : Build your distributed storage in minutes
Stars: ✭ 3,437 (+13648%)
Mutual labels:  distributed-systems, filesystem
Tifs
A distributed POSIX filesystem based on TiKV, with partition tolerance and strict consistency.
Stars: ✭ 209 (+736%)
Mutual labels:  distributed-systems, filesystem
kafka-connect-fs
Kafka Connect FileSystem Connector
Stars: ✭ 107 (+328%)
Mutual labels:  filesystem, hdfs
rorshach
A watchman for your directories. Rorshach allows you to listen to file system changes and run commands when these events occur.
Stars: ✭ 26 (+4%)
Mutual labels:  filesystem
kbdysch
A collection of user-space Linux kernel specific guided fuzzers based on LKL
Stars: ✭ 62 (+148%)
Mutual labels:  filesystem
s3-concat
Concatenate Amazon S3 files remotely using flexible patterns
Stars: ✭ 32 (+28%)
Mutual labels:  filesystem
local-hashicorp-stack
Local Hashicorp Stack for DevOps Development without Hypervisor or Cloud
Stars: ✭ 23 (-8%)
Mutual labels:  hdfs
weaver
A distributed object file system inspired by the paper Beaver which was published by Facebook Inc. at 2010.
Stars: ✭ 18 (-28%)
Mutual labels:  filesystem
rxnode
Rxnode - a small and fast wrapper around the nodejs API using RxJS.
Stars: ✭ 24 (-4%)
Mutual labels:  filesystem
Meteor-logger-file
🔖 Meteor Logging: Store application log messages into file (FS)
Stars: ✭ 24 (-4%)
Mutual labels:  filesystem
litefilesystem.js
Library with client (js) and serverside (php) to have a filesystem with previews, quotas, metadata, and multiple users with privileges.
Stars: ✭ 131 (+424%)
Mutual labels:  filesystem
Dazinator.Extensions.FileProviders
No description or website provided.
Stars: ✭ 34 (+36%)
Mutual labels:  filesystem
aether
Distributed system emulation in Common Lisp
Stars: ✭ 19 (-24%)
Mutual labels:  distributed-systems
nebula
A distributed, fast open-source graph database featuring horizontal scalability and high availability
Stars: ✭ 8,196 (+32684%)
Mutual labels:  distributed-systems
PHP-File-Cache
Light, simple and standalone PHP in-file caching class
Stars: ✭ 34 (+36%)
Mutual labels:  filesystem
rust-vfs
A virtual filesystem for Rust
Stars: ✭ 150 (+500%)
Mutual labels:  filesystem
View All Similar Projects ➔

ucz-dfs (ucz distributed file system)

A distributed file system, based on hdfs/gfs. Just for fun and learning.

CI

Work is still in progress, a lot of things are missing.

If you want to try it out, the easiest way it to spin up docker containers (docker-compose up), and bash into a running container (docker exec -it <id> bash).

Inside a running container, you can:

  1. List datanodes shell report
  2. ls shell ls /
  3. mkdir shell mkdir /foo/bar/baz
  4. Upload a file shell put build.rs /build.rs
  5. Download a file shell get /build.rs ./temp

To spin up nodes yourself:

Spin up a namenode

UDFS_CONFIG_FILE=/path/to/config.toml cargo run --bin namenode

Spin up a datanode

UDFS_CONFIG_FILE=/path/to/config.toml cargo run --bin datanode localhost:42001

Control logging

RUST_LOG=udfs=debug ...

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].