animalize / pyzstd

Introduction

Pyzstd module provides classes and functions for compressing and decompressing data, using Facebook's Zstandard (or zstd as short name) algorithm.

The API is similar to Python's bz2/lzma/zlib modules.

Includes zstd v1.5.2 source code, can also dynamically link to zstd library provided by system, and has a CFFI implementation that can work with PyPy, see this note for details.

Links

Documentation: https://pyzstd.readthedocs.io/en/latest

GitHub: https://github.com/animalize/pyzstd

Release note

0.15.3 (Aug 3, 2022)

Fix ZstdError object can't be pickled.

0.15.2 (Jan 22, 2022)

Update bundled zstd source code from v1.5.1 to v1.5.2.

0.15.1 (Dec 25, 2021)

1. Update bundled zstd source code from v1.5.0 to v1.5.1.

2. Fix ZstdFile.write() / train_dict() / finalize_dict() may use wrong length for some buffer protocol objects, see this issue.

3. Two behavior changes:

   • Setting CParameter.nbWorkers to 1 now means "1-thread multi-threaded mode", rather than "single-threaded mode".
   • If the underlying zstd library doesn't support multi-threaded compression, no longer automatically fallback to "single-threaded mode", now raise a ZstdError exception.

4. Add a module level variable zstd_support_multithread.

5. Add a setup.py option --avx2, see this note.

0.15.0 (May 18, 2021)

1. Update bundled zstd source code from v1.4.9 to v1.5.0.
2. Some improvements, no API changes.

0.14.4 (Mar 24, 2021)

1. Add a CFFI implementation that can work with PyPy.
2. Allow dynamically link to zstd library.

0.14.3 (Mar 4, 2021)

Update bundled zstd source code from v1.4.8 to v1.4.9.

0.14.2 (Feb 24, 2021)

1. Add two convenient functions: compress_stream(), decompress_stream().
2. Some improvements.

0.14.1 (Dec 19, 2020)

1. Update bundled zstd source code from v1.4.5 to v1.4.8.

   • v1.4.6 is a non-public release for Linux kernel.
   • v1.4.8 is a hotfix for v1.4.7.

2. Some improvements, no API changes.

0.13.0 (Nov 7, 2020)

1. ZstdDecompressor class: now it has the same API and behavior as BZ2Decompressor / LZMADecompressor classes in Python standard library, it stops after a frame is decompressed.

2. Add an EndlessZstdDecompressor class, it accepts multiple concatenated frames. It is renamed from previous ZstdDecompressor class, but .at_frame_edge is True when both the input and output streams are at a frame edge.

3. Rename zstd_open() function to open(), consistent with Python standard library.

4. decompress() function:

   • ~9% faster when: there is one frame, and the decompressed size was recorded in frame header.
   • raises ZstdError when input or output data is not at a frame edge. Previously, it only raise for output data is not at a frame edge.

0.12.5 (Oct 12, 2020)

No longer use Argument Clinic, now supports Python 3.5+, previously 3.7+.

0.12.4 (Oct 7, 2020)

It seems the API is stable.

0.2.4 (Sep 2, 2020)

The first version upload to PyPI.

Includes zstd v1.4.5 source code.