GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → oom-ai → oomstore

oom-ai / oomstore

Licence: Apache-2.0 license
Lightweight and Fast Feature Store Powered by Go (and Rust).

Programming Languages

go
31211 projects - #10 most used programming language
shell
77523 projects
rust
11053 projects

Labels

mlmlopsfeaturestore

Projects that are alternatives of or similar to oomstore

vertex-ai-samples
Sample code and notebooks for Vertex AI, the end-to-end machine learning platform on Google Cloud
Stars: ✭ 270 (+255.26%)
Mutual labels:  ml, mlops
Awesome Mlops
A curated list of references for MLOps
Stars: ✭ 7,119 (+9267.11%)
Mutual labels:  ml, mlops
Polyaxon
Machine Learning Platform for Kubernetes (MLOps tools for experimentation and automation)
Stars: ✭ 2,966 (+3802.63%)
Mutual labels:  ml, mlops
deepchecks
Test Suites for Validating ML Models & Data. Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.
Stars: ✭ 1,595 (+1998.68%)
Mutual labels:  ml, mlops
metaflowbot
Slack bot for monitoring your Metaflow flows!
Stars: ✭ 22 (-71.05%)
Mutual labels:  ml, mlops
Hub
Dataset format for AI. Build, manage, & visualize datasets for deep learning. Stream data real-time to PyTorch/TensorFlow & version-control it. https://activeloop.ai
Stars: ✭ 4,003 (+5167.11%)
Mutual labels:  ml, mlops
Metaflow
🚀 Build and manage real-life data science projects with ease!
Stars: ✭ 5,108 (+6621.05%)
Mutual labels:  ml, mlops
Feast
Feature Store for Machine Learning
Stars: ✭ 2,576 (+3289.47%)
Mutual labels:  ml, mlops
hi-ml
HI-ML toolbox for deep learning for medical imaging and Azure integration
Stars: ✭ 150 (+97.37%)
Mutual labels:  ml, mlops
Bentoml
Model Serving Made Easy
Stars: ✭ 3,064 (+3931.58%)
Mutual labels:  ml, mlops
cli
Polyaxon Core Client & CLI to streamline MLOps
Stars: ✭ 18 (-76.32%)
Mutual labels:  ml, mlops
VickyBytes
Subscribe to this GitHub repo to access the latest tech talks, tech demos, learning materials & modules, and developer community updates!
Stars: ✭ 48 (-36.84%)
Mutual labels:  ml, mlops
neptune-client
📒 Experiment tracking tool and model registry
Stars: ✭ 348 (+357.89%)
Mutual labels:  ml, mlops
FakeFinder
FakeFinder builds a modular framework for evaluating various deepfake detection models, offering a web application as well as API access for integration into existing workflows.
Stars: ✭ 29 (-61.84%)
Mutual labels:  mlops
ai-background-remove
Cut out objects and remove backgrounds from pictures with artificial intelligence
Stars: ✭ 70 (-7.89%)
Mutual labels:  ml
ml-lpi
Materials for ML course at Lebedev Physical Institute
Stars: ✭ 31 (-59.21%)
Mutual labels:  ml
EVHS-Programming-Club
The public GitHub repo for the EVHS Programming Club
Stars: ✭ 16 (-78.95%)
Mutual labels:  ml
ml-workflow-automation
Python Machine Learning (ML) project that demonstrates the archetypal ML workflow within a Jupyter notebook, with automated model deployment as a RESTful service on Kubernetes.
Stars: ✭ 44 (-42.11%)
Mutual labels:  mlops
aml-deploy
GitHub Action that allows you to deploy machine learning models in Azure Machine Learning.
Stars: ✭ 37 (-51.32%)
Mutual labels:  mlops
rekcurd-python
Project for serving ML module. This is a gRPC micro-framework.
Stars: ✭ 47 (-38.16%)
Mutual labels:  ml
View All Similar Projects ➔

OomStore

Lightweight and Fast Feature Store Powered by Go

CICD Version Version Platform

Quickstart · Architecture · Benchmark

Overview

oomstore allows you to:

  • Define features with YAML.
  • Store features in databases of choice.
  • Retrieve features for both online serving and offline training, fast.

Please see our docs for more details.

Features

  • 🍼 Simple. Being serverless and CLI-friendly, users can be productive in hours, not months.
  • 🔌 Composable. We support your preferred databases of choice.
  • ⚡ Fast. Benchmark shows oomstore performs QPS > 50k and latency < 0.3 ms with Redis.
  • 🌊 Streaming. We support streaming features to ensure your predictions are up-to-date.

Architecture

Architecture

See Architecture for more details.

Quickstart

  1. Install oomcli following the guide.

  2. oomcli init to initialize oomstore. Make sure there is a ~/.config/oomstore/config.yaml as below.

online-store:
  sqlite:
    db-file: /tmp/oomstore.db

offline-store:
  sqlite:
    db-file: /tmp/oomstore.db

metadata-store:
  sqlite:
    db-file: /tmp/oomstore.db
  1. oomcli apply -f metadata.yaml to register metadata. See metadata.yaml below.
kind: Entity
name: user
description: 'user ID'
groups:
- name: account
  category: batch
  description: 'user account'
  features:
  - name: state
    value-type: string
  - name: credit_score
    value-type: int64
  - name: account_age_days
    value-type: int64
  - name: 2fa_installed
    value-type: bool
- name: txn_stats
  category: batch
  description: 'user txn stats'
  features:
  - name: count_7d
    value-type: int64
  - name: count_30d
    value-type: int64
- name: recent_txn_stats
  category: stream
  snapshot-interval: 24h
  description: 'user recent txn stats'
  features:
  - name: count_10min
    value-type: int64
  1. Import CSV data to Offline Store, then sync from Offline to Online Store
oomcli import \
  --group account \
  --input-file account.csv \
  --description 'sample account data'
oomcli import \
  --group txn_stats \
  --input-file txn_stats.csv \
  --description 'sample txn stats data'
oomcli sync --group-name account --revision-id 2
oomcli sync --group-name txn_stats --revision-id 4
  1. Push stream data to both Online and Offline Store.
oomcli push --group recent_txn_stats --entity-key 1006 --feature count_10min=1
  1. Fetch features by key.
oomcli get online \
  --entity-key 1006 \
  --feature account.state,account.credit_score,account.account_age_days,account.2fa_installed,txn_stats.count_7d,txn_stats.count_30d,recent_txn_stats.count_10min
+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| user | account.state | account.credit_score | account.account_age_days | account.2fa_installed | txn_stats.count_7d | txn_stats.count_30d | recent_txn_stats.count_10min |
+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| 1006 | Louisiana     |                  710 |                       32 | false                 |                  8 |                  22 |                            1 |
+------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
  1. Generate training datasets via Point-in-Time Join.
oomcli join \
	--feature account.state,account.credit_score,account.account_age_days,account.2fa_installed,txn_stats.count_7d,txn_stats.count_30d,recent_txn_stats.count_10min \
	--input-file label.csv
+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
| entity_key |  unix_milli   | account.state | account.credit_score | account.account_age_days | account.2fa_installed | txn_stats.count_7d | txn_stats.count_30d | recent_txn_stats.count_10min |
+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+
|       1002 | 1950236233000 | Hawaii        |                  625 |                      861 | true                  |                 11 |                  36 |                              |
|       1003 | 1950411318000 | Arkansas      |                  730 |                      958 | false                 |                  0 |                  16 |                              |
|       1004 | 1950653614000 | Louisiana     |                  610 |                     1570 | false                 |                 12 |                  26 |                              |
|       1005 | 1950166137000 | South Dakota  |                  635 |                     1953 | false                 |                  7 |                  30 |                              |
|       1006 | 1950403162000 | Louisiana     |                  710 |                       32 | false                 |                  8 |                  22 |                            1 |
|       1007 | 1950160030000 | New Mexico    |                  645 |                       37 | true                  |                  5 |                  40 |                              |
|       1008 | 1950274859000 | Nevada        |                  735 |                     1627 | false                 |                 12 |                  51 |                              |
|       1009 | 1949958846000 | Kentucky      |                  650 |                       88 | true                  |                 11 |                  23 |                              |
|       1010 | 1949920686000 | Delaware      |                  680 |                     1687 | false                 |                  2 |                  39 |                              |
+------------+---------------+---------------+----------------------+--------------------------+-----------------------+--------------------+---------------------+------------------------------+

See Quickstart for more complete details.

Supported Databases

Online Store

  • Amazon DynamoDB
  • Redis
  • TiKV
  • Cassandra
  • PostgreSQL
  • MySQL
  • TiDB
  • SQLite

Offline Store

  • Snowflake
  • Amazon Redshift
  • Google BigQuery
  • PostgreSQL
  • MySQL
  • TiDB
  • SQLite

Metadata Store

  • PostgreSQL
  • MySQL
  • TiDB
  • SQLite

Community

Feel free to join the community for questions and feature requests!

Credits

oomstore is highly inspired by Feast.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].