319 Open source projects that are alternatives of or similar to serving-runtime

MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.

Spark ML Lib serving library

⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.

Summarization, translation, sentiment-analysis, text-generation and more at blazing speed using a T5 version implemented in ONNX.

DELTA is a deep learning based natural language and speech processing platform.

TensorFlow template application for deep learning

A scalable inference server for models optimized with OpenVINO™

A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Yet another yolov5, with its runtime stack for libtorch, onnx, tvm and specialized accelerators. You like torchvision's retinanet? You like yolov5? You love yolort!

volksdep is an open-source toolbox for deploying and accelerating PyTorch, ONNX and TensorFlow models with TensorRT.

This code is used to build & run a Docker container for performing predictions against a Spark ML Pipeline.

Multi Model Server is a tool for serving neural net models for inference

🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools

Serving AI/ML models in the open standard formats PMML and ONNX with both HTTP (REST API) and gRPC endpoints

The purpose of this project is to apply mediapipe to more AI chips.

Rust wrapper for Microsoft's ONNX Runtime (version 1.8)

Task-based gene regulatory network inference using single-cell or bulk gene expression data conditioned on a prior network.

A Unified Framework for Random Forest Prediction Error Estimation

Reasonable API for serving TensorFlow models using Scala

人工智障本地图片检索工具 | An EfficientNet based image retrieval tool

Exploratory Video Analytics System

The ONNX Model Zoo is a collection of pre-trained models for state of the art models in deep learning, available in the ONNX format

This is a repository for a No-Code object detection inference API using the OpenVINO. It's supported on both Windows and Linux Operating systems.

This is a repository for a semantic segmentation inference API using the OpenVINO toolkit

YOLOv5 in PyTorch > ONNX > RKNN

TensorRT person tracking RFBNet300

Poplar Advanced Runtime for the IPU

关注AI模型上线、模型部署

⚡ Open-source software for deep learning-based digital pathology

Docker scripts for building ONNX Runtime with TensorRT and OpenVINO in manylinux environment

This repository provides AI/ML service(MachineLearning model serving) modernization solution using Amazon SageMaker, AWS CDK, and AWS Serverless services.

C++ Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, OpenVINO, ncnn, MNN, SNPE, Arm NN, NNabla, ONNX Runtime, LibTorch, TensorFlow

Write fully type inferred Vuex modules

Simple and Distributed Machine Learning

How to export PyTorch models with unsupported layers to ONNX and then to Intel OpenVINO

Create a concurrent video analysis pipeline featuring multistream face and human pose detection, vehicle attribute detection, and the ability to encode multiple videos to local storage in a single stream.

This repository contains a docker image that I use to develop my artificial intelligence applications in an uncomplicated fashion. Python, TensorFlow, PyTorch, ONNX, Keras, OpenCV, TensorRT, Numpy, Jupyter notebook... 🐋🔥

A GPU-accelerated ONNX inference run-time written 100% in Rust, ready for the web

my own Fortran high level interface for HDF5

Bare-bones hdf5 serialization of Eigen matrices and vectors

No description or website provided.

Simplified Data Exchange for HPC Simulations

This fork of BVLC/Caffe is dedicated to supporting Cambricon deep learning processor and improving performance of this deep learning framework when running on Machine Learning Unit(MLU).

A fast stacked hourglass network for human pose estimation on OpenVino

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

Localize 2D image object detection in 3D Scene with Yolo in Unity Barracuda and ARFoundation.

YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with MegEngine, ONNX, TensorRT, ncnn, and OpenVINO supported. Documentation: https://yolox.readthedocs.io/

yolov5 onnx caffe

nGraph™ Backend for ONNX

PyTorch to Keras model convertor

Pyinfer is a model agnostic tool for ML developers and researchers to benchmark the inference statistics for machine learning models or functions.

tensorrt int8 量化yolov5 onnx模型

VBZ compression plugin for nanopore signal data

Powerful computer vision assisted Lego mosaic creator · Over 500,000 images created (so far!)

flowTorch - a Python library for analysis and reduced-order modeling of fluid flows

Observe workers as they pass in front of a camera to determine if they have adequate safety protection.

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX. The repository combines a class agnostic object localizer to first detect the objects in the image, and next a ResNet50 model trained on ImageNet is used to label each box.

Chainer implementation of Dense Fusion