================================================================= sqlitedict -- persistent dict, backed-up by SQLite and pickle

|Travis|_ |License|_

.. |Travis| image:: https://travis-ci.org/RaRe-Technologies/sqlitedict.svg?branch=master .. |Downloads| image:: https://img.shields.io/pypi/dm/sqlitedict.svg .. |License| image:: https://img.shields.io/pypi/l/sqlitedict.svg .. _Travis: https://travis-ci.org/RaRe-Technologies/sqlitedict .. _Downloads: https://pypi.python.org/pypi/sqlitedict .. _License: https://pypi.python.org/pypi/sqlitedict

A lightweight wrapper around Python's sqlite3 database with a simple, Pythonic dict-like interface and support for multi-thread access:

.. code-block:: python

from sqlitedict import SqliteDict mydict = SqliteDict('./my_db.sqlite', autocommit=True) mydict['some_key'] = 'any_picklable_object' print(mydict['some_key']) # prints the new value any_picklable_object for key, value in mydict.iteritems(): ... print(key, value) some_key any_picklable_object print(len(mydict)) # etc... all dict functions work 1 mydict.close()

Pickle is used internally to (de)serialize the values. Keys are arbitrary strings, values arbitrary pickle-able objects.

If you don't use autocommit (default is no autocommit for performance), then don't forget to call mydict.commit() when done with a transaction:

.. code-block:: python

using SqliteDict as context manager works too (RECOMMENDED)

with SqliteDict('./my_db.sqlite') as mydict: # note no autocommit=True ... mydict['some_key'] = u"first value" ... mydict['another_key'] = range(10) ... mydict.commit() ... mydict['some_key'] = u"new value" ... # no explicit commit here with SqliteDict('./my_db.sqlite') as mydict: # re-open the same DB ... print(mydict['some_key']) # outputs 'first value', not 'new value' first value

Features

• Values can be any picklable objects (uses cPickle with the highest protocol).

• Support for multiple tables (=dicts) living in the same database file.

• Support for access from multiple threads to the same connection (needed by e.g. Pyro). Vanilla sqlite3 gives you ProgrammingError: SQLite objects created in a thread can only be used in that same thread.

  Concurrent requests are still serialized internally, so this "multithreaded support" doesn't give you any performance benefits. It is a work-around for sqlite limitations in Python.

• Support for custom serialization or compression:

.. code-block:: python

use JSON instead of pickle

import json mydict = SqliteDict('./my_db.sqlite', encode=json.dumps, decode=json.loads)

apply zlib compression after pickling

import zlib, pickle, sqlite3 def my_encode(obj): ... return sqlite3.Binary(zlib.compress(pickle.dumps(obj, pickle.HIGHEST_PROTOCOL))) def my_decode(obj): ... return pickle.loads(zlib.decompress(bytes(obj))) mydict = SqliteDict('./my_db.sqlite', encode=my_encode, decode=my_decode)

Installation

The module has no dependencies beyond Python itself. The minimum Python version is 2.5, continuously tested on Python 2.7, and above on on Github Actions <https://github.com/RaRe-Technologies/sqlitedict/actions>_.

Install or upgrade with::

pip install -U sqlitedict

or from the source tar.gz <http://pypi.python.org/pypi/sqlitedict>_::

python setup.py install

Documentation

Standard Python document strings are inside the module:

.. code-block:: python

import sqlitedict help(sqlitedict)

(but it's just dict with a commit, really).

Beware: because of Python semantics, sqlitedict cannot know when a mutable SqliteDict-backed entry was modified in RAM. For example, mydict.setdefault('new_key', []).append(1) will leave mydict['new_key'] equal to empty list, not [1]. You'll need to explicitly assign the mutated object back to SqliteDict to achieve the same effect:

.. code-block:: python

val = mydict.get('new_key', []) val.append(1) # sqlite DB not updated here! mydict['new_key'] = val # now updated

For developers

Install::

# pip install pytest coverage pytest-coverage

To perform all tests::

# mkdir -p tests/db
# pytest tests

To perform all tests with coverage::

# pytest tests --cov=sqlitedict

Comments, bug reports

sqlitedict resides on github <https://github.com/RaRe-Technologies/sqlitedict>_. You can file issues or pull requests there.

Housekeeping

.. code-block:: python

import os os.unlink('my_db.sqlite')

sqlitedict is open source software released under the Apache 2.0 license <http://opensource.org/licenses/apache2.0.php>. Copyright (c) 2011-now Radim Řehůřek <http://radimrehurek.com> and contributors.