mayeut / pybase64

Fast Base64 implementation

This project is a wrapper on libbase64.

It aims to provide a fast base64 implementation for base64 encoding/decoding.

Installation

pip install pybase64

Usage

pybase64 uses the same API as Python base64 "modern interface" (introduced in Python 2.4) for an easy integration.

To get the fastest decoding, it is recommended to use the pybase64.b64decode and validate=True when possible.

import pybase64

print(pybase64.b64encode(b'>>>foo???', altchars='_:'))
# b'Pj4_Zm9vPz8:'
print(pybase64.b64decode(b'Pj4_Zm9vPz8:', altchars='_:', validate=True))
# b'>>>foo???'

# Standard encoding helpers
print(pybase64.standard_b64encode(b'>>>foo???'))
# b'Pj4+Zm9vPz8/'
print(pybase64.standard_b64decode(b'Pj4+Zm9vPz8/'))
# b'>>>foo???'

# URL safe encoding helpers
print(pybase64.urlsafe_b64encode(b'>>>foo???'))
# b'Pj4-Zm9vPz8_'
print(pybase64.urlsafe_b64decode(b'Pj4-Zm9vPz8_'))
# b'>>>foo???'

A command-line tool is also provided. It has encode, decode and benchmark subcommands.

usage: pybase64 [-h] [-V] {benchmark,encode,decode} ...

pybase64 command-line tool.

positional arguments:
  {benchmark,encode,decode}
                        tool help
    benchmark           -h for usage
    encode              -h for usage
    decode              -h for usage

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit

Full documentation on Read the Docs.

Benchmark

Running Python 3.7.2, Apple LLVM version 10.0.0 (clang-1000.11.45.5), Mac OS X 10.14.2 on an Intel Core i7-4870HQ @ 2.50GHz

pybase64 0.5.0 (C extension active - AVX2)
bench: altchars=None, validate=False
pybase64._pybase64.encodebytes:   1734.776 MB/s (13,271,472 bytes -> 17,928,129 bytes)
pybase64._pybase64.b64encode:     4039.539 MB/s (13,271,472 bytes -> 17,695,296 bytes)
pybase64._pybase64.b64decode:     1854.423 MB/s (17,695,296 bytes -> 13,271,472 bytes)
base64.encodebytes:                 78.352 MB/s (13,271,472 bytes -> 17,928,129 bytes)
base64.b64encode:                  539.840 MB/s (13,271,472 bytes -> 17,695,296 bytes)
base64.b64decode:                  287.826 MB/s (17,695,296 bytes -> 13,271,472 bytes)
bench: altchars=None, validate=True
pybase64._pybase64.b64encode:     4156.607 MB/s (13,271,472 bytes -> 17,695,296 bytes)
pybase64._pybase64.b64decode:     4107.997 MB/s (17,695,296 bytes -> 13,271,472 bytes)
base64.b64encode:                  559.342 MB/s (13,271,472 bytes -> 17,695,296 bytes)
base64.b64decode:                  143.674 MB/s (17,695,296 bytes -> 13,271,472 bytes)
bench: altchars=b'-_', validate=False
pybase64._pybase64.b64encode:     2786.776 MB/s (13,271,472 bytes -> 17,695,296 bytes)
pybase64._pybase64.b64decode:     1124.136 MB/s (17,695,296 bytes -> 13,271,472 bytes)
base64.b64encode:                  322.427 MB/s (13,271,472 bytes -> 17,695,296 bytes)
base64.b64decode:                  205.195 MB/s (17,695,296 bytes -> 13,271,472 bytes)
bench: altchars=b'-_', validate=True
pybase64._pybase64.b64encode:     2806.271 MB/s (13,271,472 bytes -> 17,695,296 bytes)
pybase64._pybase64.b64decode:     2740.456 MB/s (17,695,296 bytes -> 13,271,472 bytes)
base64.b64encode:                  314.709 MB/s (13,271,472 bytes -> 17,695,296 bytes)
base64.b64decode:                  121.803 MB/s (17,695,296 bytes -> 13,271,472 bytes)

Changelog

1.2.2

• Fix C extension build on musl distros
• Publish musllinux wheels

1.2.1

• Publish PyPy 3.8 (pypy38_pp73) wheels

1.2.0

• Release the GIL
• Publish CPython 3.10 wheels
• Drop python 3.5 support

1.1.4

• Add macOS arm64 wheel

1.1.3

• GitHub Actions: fix build on tag

1.1.2

• Add PyPy wheels
• Add aarch64, ppc64le & s390x manylinux wheels

1.1.1

• Move CI from TravisCI/AppVeyor to GitHub Actions
• Fix publication of Linux/macOS wheels

1.1.0

• Add b64encode_as_string, same as b64encode but returns a str object instead of a bytes object
• Add b64decode_as_bytearray, same as b64decode but returns a bytarray object instead of a bytes object
• Speed-Up decoding from UCS1 strings

1.0.2

• Update base64 library
• Publish python 3.9 wheels

1.0.1

• Publish python 3.8 wheels

1.0.0

• Drop python 3.4 support
• Drop python 2.7 support

0.5.0

• Publish python 3.7 wheels
• Drop python 3.3 support

0.4.0

• Speed-up decoding when validate==False

0.3.1

• Fix deployment issues

0.3.0

• Add encodebytes function

0.2.1

• Fixed invalid results on Windows

0.2.0

• Added documentation

• Added subcommands to the main script:

  • help
  • version
  • encode
  • decode
  • benchmark

0.1.2

• Updated base64 native library

0.1.1

• Fixed deployment issues

0.1.0

• First public release