All Projects → chengdedeng → yfs

chengdedeng / yfs

Licence: Apache-2.0 License
🏠超精简分布式对象存储

Programming Languages

java
68154 projects - #9 most used programming language

Projects that are alternatives of or similar to yfs

manon
🧪 Play with SpringBoot 2, JWT, Querydsl, GraphQL, Docker, ELK, PostgreSQL, MariaDB, Redis, MongoDB, Flyway, Maven, Gradle, TestNG, JUnit5, JaCoCo, GreenMail, CI, Quality Gates, Prometheus, Gatling, etc.
Stars: ✭ 26 (-25.71%)
Mutual labels:  maven, jdk8
Seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
Stars: ✭ 13,380 (+38128.57%)
Mutual labels:  distributed-file-system, object-storage
maven-javadoc-plugin
Apache Maven Javadoc Plugin
Stars: ✭ 45 (+28.57%)
Mutual labels:  maven
TwitterApiKit
Twitter's API v2 Objectified. This will save you time from creating data objects to access twitter's API v2. This library is supported on Gradle, Maven, Java, Kotlin, and Android projects.
Stars: ✭ 23 (-34.29%)
Mutual labels:  maven
openjfx-docs
Getting started guide for JavaFX 11
Stars: ✭ 70 (+100%)
Mutual labels:  maven
cibase
This is a toolkit for CI/CD provided by Choerodon.
Stars: ✭ 12 (-65.71%)
Mutual labels:  maven
keycloak-mail-whitelisting
Keycloak extension to whitelist email adresses domain when users register
Stars: ✭ 36 (+2.86%)
Mutual labels:  maven
scalafix-maven-plugin
Enables automatic refactoring and linting of Maven projects written in Scala using Scalafix.
Stars: ✭ 15 (-57.14%)
Mutual labels:  maven
easy-java
整理java技术要点,让java更简单,更容易上手。
Stars: ✭ 23 (-34.29%)
Mutual labels:  maven
ibm-cloud-functions-serverless-ocr-openchecks
Serverless bank check deposit processing with object storage and optical character recognition using Apache OpenWhisk powered by IBM Cloud Functions. See the Tech Talk replay for a demo.
Stars: ✭ 40 (+14.29%)
Mutual labels:  object-storage
spring-websocket-template
Template project for configuring websockets with spring
Stars: ✭ 32 (-8.57%)
Mutual labels:  maven
scalor-maven-plugin
Build integrator for Java, Scala, Scala.macro, Scala.js, Scala.native, Eclipse and Maven.
Stars: ✭ 47 (+34.29%)
Mutual labels:  maven
spring4-hibernate5-example
Spring 4 and Hibernate 5 integration example using annotations.
Stars: ✭ 16 (-54.29%)
Mutual labels:  jdk8
plot-plugin
Jenkins plot plugin
Stars: ✭ 54 (+54.29%)
Mutual labels:  maven
eclipse-null-eea-augments
Eclipse External null Annotations (EEA) repository
Stars: ✭ 34 (-2.86%)
Mutual labels:  maven
docker-s3fs
S3FS Docker image
Stars: ✭ 18 (-48.57%)
Mutual labels:  object-storage
maven-checkstyle-plugin
Apache Maven Checkstyle Plugin
Stars: ✭ 42 (+20%)
Mutual labels:  maven
minio-ruby
MinIO Client SDK for Ruby
Stars: ✭ 26 (-25.71%)
Mutual labels:  object-storage
sonatype-publish-plugin
Gradle Plugin for publishing artifacts to Sonatype and Nexus
Stars: ✭ 17 (-51.43%)
Mutual labels:  maven
JavaEE
🔥⭐️👍框架(SSM/SSH)学习笔记
Stars: ✭ 33 (-5.71%)
Mutual labels:  maven

Join the chat at https://gitter.im/chengdedeng/yfs

YFS

architecture

YFS由Gateway和Group组成,Gateway主要负责路由、鉴权、流控、安全等非存储功能,Group主要负责数据存储。 YFS采用对称式架构,没有专门的元数据管理节点,元数据和文件都分散在多个Group(存储单元)。为了实现元数据的分布式一致性和分布式协调,YFS选择了Atomix,由于 Atomix不但实现了Raft,而且提供了非常丰富的高层抽象。基于Raft协议、高可用以及文件的安全等诸多因素,所以Gateway、Group建议至少3个节点。

Group中的Store会实时上报心跳给Gateway,Gateway根据上报的信息实时修改自己的路由策略。为了保证写入的数据安全,YFS认为超半数节点写入完成才算写入成功,如果采用常规的3副本模式,那么至少2个节点写入才会返回成功。 为了应对硬件、软件、运维等多方面的威胁,YFS实现了端到端的CRC保护,能够及时检查和快速恢复异常数据。

环境及编译

  1. JDK8及以上、maven3、文件系统需要支持xattr
  2. mvn package -Dmaven.test.skip=true
  3. yfs-gateway/target/yfs-gateway-.zip和yfs-store/target/yfs-store-.zip就是部署文件
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].