GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → mcmilk → zstdmt

mcmilk / zstdmt

Licence: other
Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy and Zstandard

Programming Languages

c
50402 projects - #5 most used programming language
Makefile
30231 projects

Labels

brotlisnappyzstdlz4zstandardlizardmultithreading-librarylz5skippables-frame

Projects that are alternatives of or similar to zstdmt

7 Zip Zstd
7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
Stars: ✭ 2,150 (+1909.35%)
Mutual labels:  brotli, zstd, lz4, zstandard, lizard, lz5
EasyCompressor
⚡ A compression library that implements many compression algorithms such as LZ4, Zstd, LZMA, Snappy, Brotli, GZip, and Deflate. It helps you to improve performance by reducing Memory Usage and Network Traffic for caching.
Stars: ✭ 167 (+56.07%)
Mutual labels:  brotli, snappy, zstd, lz4, zstandard
pyrus-cramjam
Thin Python wrapper to de/compression algorithms in Rust - lightweight & no dependencies
Stars: ✭ 40 (-62.62%)
Mutual labels:  brotli, snappy, zstd, lz4
Archiver
Easily create & extract archives, and compress & decompress files of various formats
Stars: ✭ 3,373 (+3052.34%)
Mutual labels:  brotli, snappy, lz4, zstandard
box
Box - Open Standard Archive Format, a zip killer.
Stars: ✭ 38 (-64.49%)
Mutual labels:  brotli, snappy, zstandard
Compress
Optimized Go Compression Packages
Stars: ✭ 2,478 (+2215.89%)
Mutual labels:  snappy, zstd, zstandard
tiny
compress data for better performance
Stars: ✭ 21 (-80.37%)
Mutual labels:  brotli, snappy, lz4
ZstdKit
An Objective-C and Swift library for Zstd (Zstandard) compression and decompression.
Stars: ✭ 22 (-79.44%)
Mutual labels:  zstd, zstandard
Lizard
Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.
Stars: ✭ 408 (+281.31%)
Mutual labels:  brotli, zstd
Django Compression Middleware
Django middleware to compress responses using several algorithms.
Stars: ✭ 23 (-78.5%)
Mutual labels:  brotli, zstd
sqlite zstd vfs
SQLite3 extension for read/write storage compression with Zstandard
Stars: ✭ 42 (-60.75%)
Mutual labels:  zstd, zstandard
Lzbench
lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors
Stars: ✭ 490 (+357.94%)
Mutual labels:  brotli, zstd
pyzstd
Python bindings to Zstandard (zstd) compression library, the API is similar to Python's bz2/lzma/zlib modules.
Stars: ✭ 4 (-96.26%)
Mutual labels:  zstd, zstandard
pgzstd
Postgres module for Zstandard compression/decompression with preset dictionary support
Stars: ✭ 31 (-71.03%)
Mutual labels:  zstd, zstandard
P7zip
A new p7zip fork with additional codecs and improvements (forked from https://sourceforge.net/projects/p7zip/).
Stars: ✭ 222 (+107.48%)
Mutual labels:  brotli, zstd
wasm
fast wasm modules
Stars: ✭ 37 (-65.42%)
Mutual labels:  snappy, lz4
ratarmount
Random Access Read-Only Tar Mount
Stars: ✭ 217 (+102.8%)
Mutual labels:  zstd, zstandard
Turbobench
Compression Benchmark
Stars: ✭ 211 (+97.2%)
Mutual labels:  brotli, zstd
ZRA
ZStandard Random Access (ZRA) allows random access inside an archive compressed using ZStandard
Stars: ✭ 21 (-80.37%)
Mutual labels:  zstd, zstandard
sparkzstd
A zstd decompressor written in golang
Stars: ✭ 45 (-57.94%)
Mutual labels:  zstd, zstandard
View All Similar Projects ➔

Multithreading Library for Brotli, Lizard, LZ4, LZ5, Snappy-c, LZFSE and Zstandard

Description

  • works with skippables frame id 0x184D2A50 (12 bytes per compressed frame)
  • brotli is supported the same way, it will encapsulate the real brotli stream within an 16 byte frame header

Generic skippable frame definition

size value description
4 bytes 0x184D2A50U magic for skippable frame
4 bytes 4 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)

Brotli frame definition

  • the frame header for brotli is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x5242U magic for brotli "BR"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

Snappy-c frame definition

  • the frame header for Snappy-c is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x5053U magic for Snappy-c "SP"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

LZFSE frame definition

  • the frame header for Lzfse is defined a bit different:
size value description
4 bytes 0x184D2A50U magic for skippable frame (like zstd)
4 bytes 8 size of skippable frame
4 bytes compressed size size of the following frame (compressed data)
2 bytes 0x464CU magic for Lzfse "LF"
2 bytes uncompressed size allocation hint for decompressor (64KB * this size)

Usage of the Testutils

Usage of the Library

/TR 2020-10-15

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].