All Projects → arun1729 → Cog

arun1729 / Cog

Licence: mit
A Persistent Embedded Graph Database for Python

Programming Languages

python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to Cog

Xodus
Transactional schema-less embedded database used by JetBrains YouTrack and JetBrains Hub.
Stars: ✭ 864 (+860%)
Mutual labels:  database, nosql, key-value, embedded-database
Neo4j
Graphs for Everyone
Stars: ✭ 9,582 (+10546.67%)
Mutual labels:  graph, database, nosql, graph-database
Arangodb
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
Stars: ✭ 11,880 (+13100%)
Mutual labels:  database, nosql, key-value, graph-database
Ejdb
🏂 EJDB 2.0 — Embeddable JSON Database engine C library. Simple XPath like query language (JQL). Websockets / Android / iOS / React Native / Flutter / Java / Dart / Node.js bindings. Docker image.
Stars: ✭ 1,187 (+1218.89%)
Mutual labels:  database, nosql, key-value
Cutedb
A slick BTree on disk based key value store implemented in pure Go
Stars: ✭ 67 (-25.56%)
Mutual labels:  database, key-value, embedded-database
Immortaldb
🔩 A relentless key-value store for the browser.
Stars: ✭ 2,962 (+3191.11%)
Mutual labels:  database, key-value, library
Bio4j
Bio4j abstract model and general entry point to the project
Stars: ✭ 113 (+25.56%)
Mutual labels:  graph, database, graph-database
Nitrite Java
Java embedded nosql document store
Stars: ✭ 538 (+497.78%)
Mutual labels:  database, nosql, embedded-database
Bitnami Docker Redis
Bitnami Redis Docker Image
Stars: ✭ 317 (+252.22%)
Mutual labels:  database, nosql, key-value
Lmdbjava
Lightning Memory Database (LMDB) for Java: a low latency, transactional, sorted, embedded, key-value store
Stars: ✭ 546 (+506.67%)
Mutual labels:  database, key-value, embedded-database
Libmdbx
One of the fastest embeddable key-value ACID database without WAL. libmdbx surpasses the legendary LMDB in terms of reliability, features and performance.
Stars: ✭ 729 (+710%)
Mutual labels:  database, nosql, key-value
Redisgraph
A graph database as a Redis module
Stars: ✭ 1,292 (+1335.56%)
Mutual labels:  graph, nosql, graph-database
docs
Source code of the ArangoDB online documentation
Stars: ✭ 18 (-80%)
Mutual labels:  nosql, key-value, graph-database
Grakn
TypeDB: a strongly-typed database
Stars: ✭ 2,947 (+3174.44%)
Mutual labels:  graph, database, graph-database
Gokv
Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)
Stars: ✭ 314 (+248.89%)
Mutual labels:  database, key-value, library
Reddit Detective
Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
Stars: ✭ 129 (+43.33%)
Mutual labels:  graph, database, graph-database
Orientdb
OrientDB is the most versatile DBMS supporting Graph, Document, Reactive, Full-Text and Geospatial models in one Multi-Model product. OrientDB can run distributed (Multi-Master), supports SQL, ACID Transactions, Full-Text indexing and Reactive Queries. OrientDB Community Edition is Open Source using a liberal Apache 2 license.
Stars: ✭ 4,394 (+4782.22%)
Mutual labels:  database, nosql, graph-database
Hive
Lightweight and blazing fast key-value database written in pure Dart.
Stars: ✭ 2,681 (+2878.89%)
Mutual labels:  database, nosql, key-value
Iowow
The skiplist based persistent key/value storage engine
Stars: ✭ 206 (+128.89%)
Mutual labels:  database, nosql, key-value
Eliasdb
EliasDB a graph-based database.
Stars: ✭ 611 (+578.89%)
Mutual labels:  graph, database, graph-database

PyPI version Python 3.8 Build Status License: MIT codecov

Cog - Embedded Graph Database for Python

ScreenShot

New release: 2.0.3,

  • Storage optimization
  • CSV loader
  • Edge scanner
  • bug fixes

Installing Cog

pip install cogdb

Cog is a persistent embedded graph database implemented purely in Python. Torque is Cog's graph query language. Cog also provides a low level API to its fast persistent key-value store.

Cog is ideal for python applications that does not require a full featured database. Cog can easily be used as a library from within a Python application.

Cog can load a graph stored as N-Triples, a serialization format for RDF. See Wikipedia, W3C for details.

In short, an N-Triple is sequence of subject, predicate and object in a single line that defines a connection between two vertices:

vertex <predicate> vertex

Learn more about RDF triples

Creating a graph

from cog.torque import Graph
g = Graph("people")
g.put("alice","follows","bob")
g.put("bob","follows","fred")
g.put("bob","status","cool_person")
g.put("charlie","follows","bob")
g.put("charlie","follows","dani")
g.put("dani","follows","bob")
g.put("dani","follows","greg")
g.put("dani","status","cool_person")
g.put("emily","follows","fred")
g.put("fred","follows","greg")
g.put("greg","status","cool_person")

Create a graph from CSV file

from cog.torque import Graph
g = Graph("books")
g.load_csv('test/test-data/books.csv', "isbn")

Torque query examples

Scan vertices

g.scan(3)

{'result': [{'id': 'bob'}, {'id': 'emily'}, {'id': 'charlie'}]}

Scan edges

g.scan(3, 'e')

{'result': [{'id': 'status'}, {'id': 'follows'}]}

Starting from a vertex, follow all outgoing edges and list all vertices

g.v("bob").out().all()

{'result': [{'id': 'cool_person'}, {'id': 'fred'}]}

starting from a vertex, follow all outgoing edges and count vertices

g.v("bob").out().count()

'2'

starting from a vertex, follow all out going edges and tag them

g.v("bob").out().tag("source").out().tag("target").all()

{'result': [{'source': 'fred', 'id': 'greg', 'target': 'greg'}]}

By tagging the vertices 'source' and 'target', the resulting graph can be visualized using Sigma JS

starting from a vertex, follow all incoming edges and list all vertices

g.v("bob").inc().all()

{'result': [{'id': 'alice'}, {'id': 'charlie'}, {'id': 'dani'}]}

Loading data from a file

Triples file

from cog.torque import Graph
g = Graph(graph_name="people")
g.load_triples("/path/to/triples.nt", "people")

Edgelist file

from cog.torque import Graph
g = Graph(graph_name="people")
g.load_edgelist("/path/to/edgelist", "people")

Low level key-value store API:

Every record inserted into Cog's key-value store is directly persisted on to disk. It stores and retrieves data based on hash values of the keys, it can perform fast look ups (O(1) avg) and fast (O(1) avg) inserts.

from cog.database import Cog

cogdb = Cog('path/to/dbdir')

# create a namespace
cogdb.create_namespace("my_namespace")

# create new table
cogdb.create_table("new_db", "my_namespace")

# put some data
cogdb.put(('key','val'))

# retrieve data 
cogdb.get('key')

# put some more data
cogdb.put(('key2','val2'))

# scan
scanner = cogdb.scanner()
for r in scanner:
    print r
    
# delete data
cogdb.delete('key1')

Config

If no config is provided when creating a Cog instance, it will use the defaults:

COG_PATH_PREFIX = "/tmp"
COG_HOME = "cog-test"

Example updating config

from cog import config
config.COG_HOME = "app1_home"
data = ('user_data:id=1','{"firstname":"Hari","lastname":"seldon"}')
cog = Cog(config)
cog.create_namespace("test")
cog.create_table("db_test", "test")
cog.put(data)
scanner = cog.scanner()
for r in scanner:
    print r

Performance

Put and Get calls performance:

put ops/second: 18968

get ops/second: 39113

The perf test script is included with the tests: insert_bench.py and get_bench.py

INDEX_LOAD_FACTOR on an index determines when a new index file is created, Cog uses linear probing to resolve index collisions. Higher INDEX_LOAD_FACTOR leads slightly lower performance on operations on index files that have reached the target load.

Put and Get performance profile

Put Perf Get Perf

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].