All Projects → jonhoo → shortcut

jonhoo / shortcut

Licence: Apache-2.0, MIT licenses found Licenses found Apache-2.0 LICENSE-APACHE MIT LICENSE-MIT
Rust crate providing an indexed, queryable column-based storage system

Programming Languages

rust
11053 projects

Projects that are alternatives of or similar to shortcut

Js Search
JS Search is an efficient, client-side search library for JavaScript and JSON objects
Stars: ✭ 1,920 (+6757.14%)
Mutual labels:  indexing
Sfa
Scalable Time Series Data Analytics
Stars: ✭ 222 (+692.86%)
Mutual labels:  indexing
http-server-pwa
👾 http-server alike but for serving and rendering PWA: pwa-server
Stars: ✭ 14 (-50%)
Mutual labels:  indexing
Pathivu
An efficient log ingestion and log aggregation system https://pathivu.io/
Stars: ✭ 146 (+421.43%)
Mutual labels:  indexing
Alfanous
Alfanous is an Arabic search engine API provides the simple and advanced search in Quran , more features and many interfaces...
Stars: ✭ 209 (+646.43%)
Mutual labels:  indexing
Hyperspace
An open source indexing subsystem that brings index-based query acceleration to Apache Spark™ and big data workloads.
Stars: ✭ 246 (+778.57%)
Mutual labels:  indexing
Btree4j
Disk-based B+-tree written in Pure Java
Stars: ✭ 122 (+335.71%)
Mutual labels:  indexing
quranize
transform transliteration to Quran text
Stars: ✭ 13 (-53.57%)
Mutual labels:  indexing
Paperwork
Personal document manager (Linux/Windows) -- Moved to Gnome's Gitlab
Stars: ✭ 2,392 (+8442.86%)
Mutual labels:  indexing
secondary
Redis Secondary Indexing Module, been suspended see: https://github.com/RediSearch/RediSearch/
Stars: ✭ 33 (+17.86%)
Mutual labels:  indexing
Tg Index
Python web app to index telegram channel and serve its files for download.
Stars: ✭ 157 (+460.71%)
Mutual labels:  indexing
Examine
A .NET indexing and search engine powered by Lucene.Net
Stars: ✭ 208 (+642.86%)
Mutual labels:  indexing
Foundatio.Repositories
Generic repositories
Stars: ✭ 74 (+164.29%)
Mutual labels:  indexing
Recipe
RECIPE : high-performance, concurrent indexes for persistent memory (SOSP 2019)
Stars: ✭ 145 (+417.86%)
Mutual labels:  indexing
gdrive-index
An index server for Google Drive
Stars: ✭ 107 (+282.14%)
Mutual labels:  indexing
Dashing
Fast and accurate genomic distances using HyperLogLog
Stars: ✭ 131 (+367.86%)
Mutual labels:  indexing
Esbulk
Bulk indexing command line tool for elasticsearch
Stars: ✭ 235 (+739.29%)
Mutual labels:  indexing
fcgi-function
A cross-platform module to writing C/C++ service for nginx.
Stars: ✭ 33 (+17.86%)
Mutual labels:  indexing
btree
A persistent B+Tree (clustered index) implementation in Rust.
Stars: ✭ 167 (+496.43%)
Mutual labels:  indexing
kmer-db
Kmer-db is a fast and memory-efficient tool for large-scale k-mer analyses (indexing, querying, estimating evolutionary relationships, etc.).
Stars: ✭ 68 (+142.86%)
Mutual labels:  indexing

shortcut

Crates.io Documentation Build Status

This crate provides an indexed, queryable column-based storage system.

The storage system is, fundamentally, row-based storage, where all rows have the same number of columns. All columns are the same "type", but given that they can be enum types, you can effectively use differently typed values. Data is stored in a BTreeMap<usize, Vec<T>>, where the outermost BTreeMap is dynamically sized (and may be re-allocated as more rows come in), whereas the innermost Vec is expected to never change. The map index is an autoincremented row identifier similar to the one used by SQLite: https://www.sqlite.org/lang_createtable.html#rowid.

What makes this crate interesting is that it also allows you to place indices on columns for fast lookups. These indices are automatically updated whenever the dataset changes, so that queries continue to return correct results. Indices should conform to either the EqualityIndex trait or the RangeIndex trait. As you would expect, the former allows speeding up exact lookups, whereas the latter can also perform efficient range queries.

Queries are performed over the dataset by calling find with a set of Conditions that will be ANDed together. OR is currently not supported --- issue multiple quieries instead. Each Condition represents a value comparison against the value in a single column. The system automatically picks what index to use to satisfy the query, using a heuristic based on the expected number of rows returned for that column for each index.

Known limitations

  • The set of match operations is currently fairly limited.
  • The system currently provides an add/remove-only abstraction (i.e., no edit).
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].