All Categories → No Category → hadoop-hdfs

Top 3 hadoop-hdfs open source projects

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

✭ 13,380

go java kubernetes distributed-systems s3 posix fuse replication hdfs distributed-storage object-storage s3-storage cloud-drive distributed-file-system erasure-coding blob-storage seaweedfs hadoop-hdfs tiered-file-system

datalake-etl-pipeline

Simplified ETL process in Hadoop using Apache Spark. Has complete ETL pipeline for datalake. SparkSession extensions, DataFrame validation, Column extensions, SQL functions, and DataFrame transformations

✭ 39

python big-data spark apache-spark hadoop etl xml xml-parsing pyspark data-pipeline datalake hadoop-mapreduce spark-sql etl-framework hadoop-hdfs etl-pipeline etl-components

marayarn

Marathon on yarn

✭ 13

java javascript HTML yarn marathon hadoop-hdfs

1-3 of 3 hadoop-hdfs projects