All Projects → powturbo → Turbo Run Length Encoding

powturbo / Turbo Run Length Encoding

TurboRLE-Fastest Run Length Encoding

Programming Languages

java
68154 projects - #9 most used programming language
c
50402 projects - #5 most used programming language

Projects that are alternatives of or similar to Turbo Run Length Encoding

Turbo-Histogram
Fastest Histogram Construction
Stars: ✭ 44 (-79.25%)
Mutual labels:  benchmark, sse, simd, avx2
simd-byte-lookup
SIMDized check which bytes are in a set
Stars: ✭ 23 (-89.15%)
Mutual labels:  sse, simd, avx2
Unisimd Assembler
SIMD macro assembler unified for ARM, MIPS, PPC and x86
Stars: ✭ 63 (-70.28%)
Mutual labels:  simd, sse, avx2
Simd
C++ image processing and machine learning library with using of SIMD: SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX-512, VMX(Altivec) and VSX(Power7), NEON for ARM.
Stars: ✭ 1,263 (+495.75%)
Mutual labels:  simd, sse, avx2
Simde
Implementations of SIMD instruction sets for systems which don't natively support them.
Stars: ✭ 1,012 (+377.36%)
Mutual labels:  simd, sse, avx2
ternary-logic
Support for ternary logic in SSE, XOP, AVX2 and x86 programs
Stars: ✭ 21 (-90.09%)
Mutual labels:  sse, simd, avx2
Quadray Engine
Realtime raytracer using SIMD on ARM, MIPS, PPC and x86
Stars: ✭ 13 (-93.87%)
Mutual labels:  simd, sse, avx2
Turbo-Transpose
Transpose: SIMD Integer+Floating Point Compression Filter
Stars: ✭ 50 (-76.42%)
Mutual labels:  sse, simd, avx2
Libxsmm
Library for specialized dense and sparse matrix operations, and deep learning primitives.
Stars: ✭ 518 (+144.34%)
Mutual labels:  simd, sse, avx2
Base64simd
Base64 coding and decoding with SIMD instructions (SSE/AVX2/AVX512F/AVX512BW/AVX512VBMI/ARM Neon)
Stars: ✭ 115 (-45.75%)
Mutual labels:  simd, sse, avx2
Vc
SIMD Vector Classes for C++
Stars: ✭ 985 (+364.62%)
Mutual labels:  simd, sse, avx2
Umesimd
UME::SIMD A library for explicit simd vectorization.
Stars: ✭ 66 (-68.87%)
Mutual labels:  simd, benchmark, avx2
Boost.simd
Boost SIMD
Stars: ✭ 238 (+12.26%)
Mutual labels:  simd, sse, avx2
Libsimdpp
Portable header-only C++ low level SIMD library
Stars: ✭ 914 (+331.13%)
Mutual labels:  simd, sse, avx2
cpuwhat
Nim utilities for advanced CPU operations: CPU identification, ISA extension detection, bindings to assorted intrinsics
Stars: ✭ 25 (-88.21%)
Mutual labels:  sse, simd, avx2
Fastnoisesimd
C++ SIMD Noise Library
Stars: ✭ 542 (+155.66%)
Mutual labels:  simd, sse, avx2
Directxmath
DirectXMath is an all inline SIMD C++ linear algebra library for use in games and graphics apps
Stars: ✭ 859 (+305.19%)
Mutual labels:  simd, sse, avx2
Xsimd
C++ wrappers for SIMD intrinsics and parallelized, optimized mathematical functions (SSE, AVX, NEON, AVX512)
Stars: ✭ 964 (+354.72%)
Mutual labels:  simd, sse
Toys
Storage for my snippets, toy programs, etc.
Stars: ✭ 187 (-11.79%)
Mutual labels:  sse, avx2
Simdjson
Parsing gigabytes of JSON per second
Stars: ✭ 15,115 (+7029.72%)
Mutual labels:  simd, avx2

TurboRLE: Turbo Run Length Encoding Build Status

Efficient and fastest Run Length Encoding library
  • 🆕 The fastest now up to 50% more faster incl. SSE/AVX2 + improved compression

  • 🆕 ARM NEON support

  • 100% C (C++ compatible headers), without inline assembly

  • Most efficient compression

  • No other RLE compress or decompress faster with better compression

  • ✨ faster compression and 2x faster decompression with 👍 SIMD

  • 👍 Java Critical Natives Interface. Access TurboRLE incl. SIMD! from Java

  • Compress better and up to 12 times faster and decompress up to 6 times faster than other fast RLEs

  • Can be faster than memcpy!

  • 👍 ZERO! byte overhead

  • No modification of the raw data, preserving compressibility for further postprocessing (ex. entropy coding)

  • Order preserving

TurboRLE
  • TRLE: TurboRLE - Efficient and fast Run Length Encoding
  • SRLE: TurboRLE Escape - Fast Run Length Encoding with automatic escape determination

Benchmark:

  • Single thread with TurboBench
  • Realistic and practical benchmark with large files and different distributions
External functions benchmarked
  • MRLE: Mespotine RLE: Run Length Encoding
  • RLE8: A fast 8 bit Run Length Encoding (SSE/AVX2)

CPU: Skylake i7-6700 3.4GHz, gcc 8.3 (2019-08)

(bold = pareto) MB=1.000.000

C Size ratio% C MB/s D MB/s Name / 2019-08
2623680 0.6 2089 11385 trle
4148455 1.0 2095 12521 srle 0 (auto escape)
4482384 1.1 378 12302 mrle
4744806 1.2 11307 13205 srle 8
5901235 1.5 890 11998 rle8 1
8431844 2.1 7368 12693 srle 16
13722311 3.4 11090 13188 srle 32
19839711 4.9 16269 13733 srle 64
403920054 100.0 13978 14001 memcpy
  • Checkers program "End Game Table Base": 1034.db
C Size ratio% C MB/s D MB/s Name / 2019-08
73108990 17.4 774 3020 trle
84671759 20.2 764 5113 srle 0
88055360 21.0 262 1491 mrle
88666372 21.2 456 2669 rle8 1
92369164 22.0 1101 5967 srle 8
113561548 27.1 2028 7114 srle 16
136918311 32.7 3588 9026 srle 32
165547365 39.5 5972 10120 srle 64
419225625 100.0 13938 14017 memcpy
C Size ratio% C MB/s D MB/s Name / 2019-08
375094084 37.5 472 1743 trle
415597104 41.6 461 3563 srle 0
419263924 41.9 568 4310 srle 8
487430623 48.7 1347 6287 srle 16
549202860 54.9 2780 8238 srle 32
576619941 57.7 207 553 mrle
577685250 57.8 260 675 rle8 1
605759578 60.6 5356 9471 srle 64
1000000008 100.0 13931 13926 memcpy

CPU: ARM A73-ODROID-N2 1.8GHz, gcc 8.3 (2019-08)

C Size ratio% C MB/s D MB/s Name / 2019-08
2623680 0.6 594 3399 trle
4148455 1.0 589 3799 srle 0
4482384 1.1 144 3712 mrle
4744806 1.2 3365 3790 srle 8
8431844 2.1 1313 3784 srle 16
13722311 3.4 1771 4573 srle 32
19839711 4.9 4184 6020 srle 64
403920054 100.0 3345 3376 memcpy
C Size ratio% C MB/s D MB/s Name / 2019-08
375094084 37.5 184 722 trle
415597104 41.6 180 681 srle 0
419263924 41.9 244 693 srle 8
487430623 48.7 455 1203 srle 16
549202860 54.9 825 2094 srle 32
576619941 57.7 94 249 mrle
605759578 60.6 1677 3662 srle 64
1000000008 100.0 3830 3841 memcpy

  • Post-processing: Entropy Coding after Run Length Encoding
    Direct entropy encoding after "trle" (no additional "move to front" or other transformation)
C Size ratio% C MB/s D MB/s Name / CPU Skylake 3.4 GHz (2019-06)
180510948 18.1 154 132 trle + TurboRC o0 (order 0 bitwise Range Coder)
187099490 18.7 23 2560 trle + TurboHF 0 (Huffman Coding)
192420471 19.2 1527 3834 trle + TurboANX 12 (Asymmetric Numeral Systems)
193455670 19.3 2192 2986 trle + TurboHF 12
197974078 19.8 1078 1406 trle + fse (Finite State Entropy)
229693376 23.0 126 106 rle8 + TurboRC o0
254312056 25.4 119 105 mrle + TurboRC o0

for more info, see also: Entropy Coding Benchmark

Compile:

	git clone git://github.com/powturbo/Turbo-Run-Length-Encoding.git
    cd Turbo-Run-Length-Encoding
Linux + Windows MingW
	make
    or
	make AVX2=1
Windows Visual C++
	nmake /f makefile.vs
    or
	nmake AVX2=1 /f makefile.vs

Testing Run Length Encoding libraries:

    ./trle file
    ./trle -e# file

	# = function id (see file trle.c)

Environment:

OS/Compiler (32 + 64 bits):
  • Windows: MinGW-w64 makefile
  • Windows: Visual c++ (>=VS2008) - makefile.vs (for nmake)
  • Windows: Visual Studio project file - vs/vs2017
  • Linux amd64: GNU GCC (>=4.6)
  • Linux amd64: Clang (>=3.2)
  • Linux arm64: 64 bits aarch64 ARMv8: gcc (>=6.3)
  • Linux arm64: 64 bits aarch64 ARMv8: clang
  • MaxOS: XCode (>=9)
  • PowerPC ppc64le (incl. SIMD): gcc (>=8.0)

Run Length Encoding References:

Last update: 25 FEB 2020

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].