GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → richox → Orz

richox / Orz

Licence: mit
a high performance, general purpose data compressor written in rust

Programming Languages

rust
11053 projects

Labels

compressionhigh-performance

Projects that are alternatives of or similar to Orz

Fflate
High performance (de)compression in an 8kB package
Stars: ✭ 547 (+7.47%)
Mutual labels:  compression, high-performance
Embedded Neural Network
collection of works aiming at reducing model sizes or the ASIC/FPGA accelerator for machine learning
Stars: ✭ 495 (-2.75%)
Mutual labels:  compression
Zstd Jni
JNI binding for Zstd
Stars: ✭ 424 (-16.7%)
Mutual labels:  compression
Symbolics.jl
A fast and modern CAS for a fast and modern language.
Stars: ✭ 435 (-14.54%)
Mutual labels:  high-performance
Javafastpfor
A simple integer compression library in Java
Stars: ✭ 426 (-16.31%)
Mutual labels:  compression
Compression
Data compression in TensorFlow
Stars: ✭ 458 (-10.02%)
Mutual labels:  compression
Httpteleport
Transfer 10Gbps http traffic over 1Gbps networks :)
Stars: ✭ 422 (-17.09%)
Mutual labels:  compression
Rust Brotli
Brotli compressor and decompressor written in rust that optionally avoids the stdlib
Stars: ✭ 504 (-0.98%)
Mutual labels:  compression
Sled
the champagne of beta embedded databases
Stars: ✭ 5,423 (+965.42%)
Mutual labels:  high-performance
Compactor
A user interface for Windows 10 filesystem compression
Stars: ✭ 445 (-12.57%)
Mutual labels:  compression
Dwarfs
A fast high compression read-only file system
Stars: ✭ 444 (-12.77%)
Mutual labels:  compression
Enzyme
High-performance automatic differentiation of LLVM.
Stars: ✭ 418 (-17.88%)
Mutual labels:  high-performance
Anakin
High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.
Stars: ✭ 488 (-4.13%)
Mutual labels:  high-performance
Draco
Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.
Stars: ✭ 4,611 (+805.89%)
Mutual labels:  compression
Xxl Rpc
A high performance, distributed RPC framework.（分布式服务框架XXL-RPC）
Stars: ✭ 493 (-3.14%)
Mutual labels:  high-performance
Fastd
🚀 A high performance PHP API framework.
Stars: ✭ 423 (-16.9%)
Mutual labels:  high-performance
Tensorbase
TensorBase BE is building a high performance, cloud neutral bigdata warehouse for SMEs fully in Rust.
Stars: ✭ 440 (-13.56%)
Mutual labels:  high-performance
Aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
Stars: ✭ 453 (-11%)
Mutual labels:  compression
Fast float
Fast and exact implementation of the C++ from_chars functions for float and double types: 4x faster than strtod
Stars: ✭ 510 (+0.2%)
Mutual labels:  high-performance
Pgm Index
🏅State-of-the-art learned data structure that enables fast lookup, predecessor, range searches and updates in arrays of billions of items using orders of magnitude less space than traditional indexes
Stars: ✭ 499 (-1.96%)
Mutual labels:  compression
View All Similar Projects ➔

Orz

orz -- a general purpose data compressor written in rust.

LICENSE Build Status

orz is an optimized ROLZ (reduced offset Lempel-Ziv) general purpose data compressor. input data is encoded as ROLZ-matches (reduced-offsets and match lengths), 2-byte words, and single bytes. then all encoded symbols are processed with a symbol ranking (aka Move-to-Front) transformer and a static huffman coder.

benefited from the ROLZ algorithm, orz compresses times faster than many other LZ-based compressors which has same compression ratio, and decompression speed is still very acceptable.

orz is completely implemented in rust. clone the repo and run cargo build --release to have an executable orz binary.

installation

you can install orz with cargo:

cargo install orz --git https://github.com/richox/orz --tag v1.6.2

usage

for compression:

orz encode <source-file-input> <compressed-file-output>

for decompression:

orz decode <compressed-file-input> <source-file-output>

for more details, see orz --help

benchmarks

benchmark for 100MB of Large Text Compression Benchmark (enwik8, see http://mattmahoney.net/dc/text.html):

name compressed size encode time decode time
xz -6 26,375,764 71.601s 1.563s
orz -l2 27,100,084 8.151s 1.299s
orz -l1 27,381,156 6.812s 1.314s
zstd -19 27,659,082 52.844s 0.244s
orz -l0 28,025,726 5.630s 1.327s
bzip2 -9 29,008,758 8.385s 3.922s
brotli -9 29,685,672 35.458s 0.340s
brotli -8 30,326,580 20.501s 0.311s
zstd -15 30,328,568 23.030s 0.195s
brotli -7 31,057,759 12.267s 0.307s
zstd -11 31,230,229 8.992s 0.206s
lzfse 36,157,828 1.976s 0.176s
gzip -6 36,518,322 4.948s 0.672s

benchmark for 400MB of text data of Global Data Compression Competition (TS40.txt, see https://globalcompetition.compression.ru/#leaderboards): | name |compressed size|encode time|decode time| |-----------|---------------|-----------|-----------| | xz -6 | 108,677,096 | 335.738s | 5.887s | | bzip2 -9 | 109,502,210 | 35.331s | 15.986s | |orz -l2| 111,844,429 | 31.955s | 4.890s | | zstd -19 | 112,679,835 | 252.155s | 1.050s | |orz -l1| 113,065,821 | 26.168s | 4.799s | |orz -l0| 116,003,142 | 20.172s | 4.785s | | zstd -15 | 123,100,586 | 110.805s | 0.878s | | brotli -9 | 124,453,389 | 144.100s | 1.422s | | brotli -8 | 126,791,079 | 78.620s | 1.281s | | zstd -11 | 127,940,149 | 40.962s | 0.827s | | brotli -7 | 129,425,945 | 45.338s | 1.245s | | gzip -6 | 146,656,915 | 25.237s | 2.662s | | lzfse | 147,579,002 | 8.220s | 0.832s |

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].