All Projects → master131 → BrotliSharpLib

master131 / BrotliSharpLib

Licence: MIT License
Full C# port of Brotli compression algorithm

Programming Languages

C#
18002 projects

Projects that are alternatives of or similar to BrotliSharpLib

pyrus-cramjam
Thin Python wrapper to de/compression algorithms in Rust - lightweight & no dependencies
Stars: ✭ 40 (-48.05%)
Mutual labels:  compression, decompression, brotli
EasyCompressor
⚡ A compression library that implements many compression algorithms such as LZ4, Zstd, LZMA, Snappy, Brotli, GZip, and Deflate. It helps you to improve performance by reducing Memory Usage and Network Traffic for caching.
Stars: ✭ 167 (+116.88%)
Mutual labels:  compression, decompression, brotli
Rust Brotli
Brotli compressor and decompressor written in rust that optionally avoids the stdlib
Stars: ✭ 504 (+554.55%)
Mutual labels:  compression, decompression, brotli
Huffman-Coding
A C++ compression program based on Huffman's lossless compression algorithm and decoder.
Stars: ✭ 81 (+5.19%)
Mutual labels:  compression, decompression
Util
A collection of useful utility functions
Stars: ✭ 201 (+161.04%)
Mutual labels:  compression, decompression
Turbobench
Compression Benchmark
Stars: ✭ 211 (+174.03%)
Mutual labels:  compression, brotli
Libmspack
A library for some loosely related Microsoft compression formats, CAB, CHM, HLP, LIT, KWAJ and SZDD.
Stars: ✭ 104 (+35.06%)
Mutual labels:  compression, decompression
box
Box - Open Standard Archive Format, a zip killer.
Stars: ✭ 38 (-50.65%)
Mutual labels:  compression, brotli
gorilla
An effective time-series data compression/decompression method based on Facebook's Gorilla.
Stars: ✭ 51 (-33.77%)
Mutual labels:  compression, decompression
gzipped
Replacement for golang http.FileServer which supports precompressed static assets.
Stars: ✭ 86 (+11.69%)
Mutual labels:  compression, brotli
power-gzip
POWER9 gzip engine documentation and code samples
Stars: ✭ 16 (-79.22%)
Mutual labels:  compression, decompression
Sdefl
Small inflate/deflate implementation in ~300 LoC of ANSI C
Stars: ✭ 120 (+55.84%)
Mutual labels:  compression, decompression
Libbrotli
meta project to build libraries from the brotli source code
Stars: ✭ 110 (+42.86%)
Mutual labels:  compression, brotli
Minlzma
The Minimal LZMA (minlzma) project aims to provide a minimalistic, cross-platform, highly commented, standards-compliant C library (minlzlib) for decompressing LZMA2-encapsulated compressed data in LZMA format within an XZ container, as can be generated with Python 3.6, 7-zip, and xzutils
Stars: ✭ 236 (+206.49%)
Mutual labels:  compression, decompression
Docker Nginx Brotli
Stable nginx with google brotli compression module
Stars: ✭ 107 (+38.96%)
Mutual labels:  compression, brotli
lambdafs
Efficient (de)compression package for AWS Lambda
Stars: ✭ 24 (-68.83%)
Mutual labels:  decompression, brotli
zstd-rs
zstd-decoder in pure rust
Stars: ✭ 148 (+92.21%)
Mutual labels:  compression, decompression
django-brotli
Django middleware that compresses response using brotli algorithm.
Stars: ✭ 16 (-79.22%)
Mutual labels:  compression, brotli
raisin
A simple lightweight set of implementations and bindings for compression algorithms written in Go.
Stars: ✭ 17 (-77.92%)
Mutual labels:  compression, decompression
Numcompress
Python package to compress numerical series & numpy arrays into strings
Stars: ✭ 68 (-11.69%)
Mutual labels:  compression, decompression

BrotliSharpLib

BrotliSharpLib is a full C# port of the brotli library/compression code by Google. It is intended to be a mostly 1:1 conversion of the original C code. All code is correct as of v0.6.0 of the reference implementation.

The projects uses a minimal set of APIs to ensure compatibility with a wide range of frameworks including .NET Standard and .NET Core. It also supports little-endian and big-endian architectures and is optimised for x86, x64 and ARM processors.

BrotliSharpLib is licensed under MIT.

Installation

BrotliSharpLib can be installed via the NuGet package here.

Install-Package BrotliSharpLib

Usage

Generic/basic usage:

/** Decompression **/
byte[] brotliCompressedData = ...; // arbritary data source
byte[] uncompressedData = Brotli.DecompressBuffer(brotliCompressedData, 0, brotliCompressedData.Length /**, customDictionary **/);

/** Compression **/
byte[] uncompressedData = ...; // arbritary data source

// By default, brotli uses a quality value of 11 and window size of 22 if the parameters are omitted.
byte[] compressedData = Brotli.CompressBuffer(uncompressedData, 0, uncompressedData.Length /**, quality, windowSize, customDictionary **/);

Stream usage:

/** Decompression **/
using (var ms = new MemoryStream())
using (var bs = new BrotliStream(compressedStream, CompressionMode.Decompress))
{
    bs.CopyTo(ms);
}

/** Compression **/
using (var fs = File.OpenRead(filePath))
using (var ms = new MemoryStream())
{
    using (var bs = new BrotliStream(ms, CompressionMode.Compress))
    {
        // By default, BrotliSharpLib uses a quality value of 1 and window size of 22 if the methods are not called.
        /** bs.SetQuality(quality); **/
        /** bs.SetWindow(windowSize); **/
        /** bs.SetCustomDictionary(customDict); **/
        fs.CopyTo(bs);
    }

    byte[] compressed = ms.ToArray();
}

Real-life example: The following allows for acceptance and decompression of brotli encoded web content via a HttpClient and falls back to gzip or deflate when required.

static class HttpClientEx
{
    private class BrotliCompressionHandler : DelegatingHandler
    {
        protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
        {
            request.Headers.AcceptEncoding.Add(new StringWithQualityHeaderValue("br"));
            var response = await base.SendAsync(request, cancellationToken);
            IEnumerable<string> ce;
            if (response.Content.Headers.TryGetValues("Content-Encoding", out ce) && ce.First() == "br")
            {
                var buffer = await response.Content.ReadAsByteArrayAsync();
                response.Content = new ByteArrayContent(Brotli.DecompressBuffer(buffer, 0, buffer.Length));
            }
            return response;
        }
    }

    public static HttpClient Create()
    {
        var handler = new HttpClientHandler();
        if (handler.SupportsAutomaticDecompression)
            handler.AutomaticDecompression = System.Net.DecompressionMethods.Deflate | System.Net.DecompressionMethods.GZip;
        return HttpClientFactory.Create(handler, new BrotliCompressionHandler());
    }
}

Performance

Considerations for Build

For optimal performance, ensure to build BrotliSharpLib in Release mode to enable all possible JIT optimisations.

Performance can also be further improved by building BrotliSharpLib using .NET Framework 4.5 or above (or any framework that supports AggressiveInlining). Selecting a specific target platform (instead of AnyCPU) where possible can also further improve performance. All of this however, is completely optional as BrotliSharpLib is designed to run in a wide range of contexts and configurations regardless.

Benchmark

The following are benchmark results using DotNetBenchmark with BrotliSharpLib (v0.2.1) and Google's C# implementation built against .NET Framework 4.6.1. The original C version was compiled in Release mode using Visual Studio 2017 (v141) as a 64-bit Windows executable.

BenchmarkDotNet=v0.10.6, OS=Windows 10 Redstone 2 (10.0.15063)
Processor=Intel Core i5-6600K CPU 3.50GHz (Skylake), ProcessorCount=4
Frequency=3421875 Hz, Resolution=292.2374 ns, Timer=TSC
  [Host]       : Clr 4.0.30319.42000, 64bit RyuJIT-v4.7.2046.0
  RyuJitX64    : Clr 4.0.30319.42000, 64bit RyuJIT-v4.7.2046.0
Runtime=Clr  

Decompression

File: UPX v3.91 (Windows Executable)

Method Mean
GoogleImpl 12.75 ms
BrotliSharpLib 11.63 ms
Original C 11.17 ms

As seen above, BrotliSharpLib performs close to the original C version in terms of decompression.

Compression

File: plrabn12.txt

Method Quality Mean
BrotliSharpLib 1 9.132 ms
Original C 1 9.570 ms
BrotliSharpLib 6 58.720 ms
Original C 6 36.540 ms
BrotliSharpLib 9 116.318 ms
Original C 9 73.080 ms
BrotliSharpLib 11 1822.702 ms
Original C 11 877.58 ms

While BrotliSharpLib performs comparatively at lower quality levels, it performs up to three times worse at level 11. Future versions of the port will hopefully bring this down.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].