Top 162 inference open source projects

DAWNBench: An End-to-End Deep Learning Benchmark and Competition

Model Zoo for Intel® Architecture: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors

TypeDB: a strongly-typed database

Adlik: Toolkit for Accelerating Deep Learning Inference

This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.

Explore the Capabilities of the TensorRT Platform

A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.

Save, Load Frozen Graph and Run Inference From Frozen Graph in TensorFlow 1.x and 2.x

通用深度学习推理服务，可在生产环境中快速上线由TensorFlow、PyTorch、Caffe框架训练出的深度学习模型。

ELFI - Engine for Likelihood-Free Inference

volksdep is an open-source toolbox for deploying and accelerating PyTorch, ONNX and TensorFlow models with TensorRT.

Serving PyTorch 1.0 Models as a Web Server in C++

Train and run Pytorch models on Apache Spark.

PyTorch C++ inference with LibTorch

An easy to use PyTorch to TensorRT converter

This is a repository for an nocode object detection inference API using the Yolov4 and Yolov3 Opencv.

Yolov3 Object Detection implemented as APIs, using TensorFlow and Flask

OpenVINO™ Toolkit repository

Running BERT without Padding

Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Gaussian Process Motion Planner 2

Deep Learning based Automatic Speech Recognition with attention for the Nvidia Jetson.

This is a repository for an object detection inference API using the Tensorflow framework.

Named Entity Recognition with BERT using TensorFlow 2.0

Machine Learning inference engine for Microcontrollers and Embedded devices

TensorFlow template application for deep learning

Cross-platform, customizable ML solutions for live and streaming media.

ncnn is a high-performance neural network inference framework optimized for the mobile platform

This repository provides you with a easy to use labeling tool for State-of-the-art Deep Learning training purposes.

Summarization, translation, sentiment-analysis, text-generation and more at blazing speed using a T5 version implemented in ONNX.

A library for high performance deep learning inference on NVIDIA GPUs.

Automatic Haskell type inference from JSON input

Highly optimized inference engine for Binarized Neural Networks

benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.

MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.

Acceleration package for neural networks on multi-core CPUs

An Open Source Deep Learning Inference Engine Based on FPGA

Yet another yolov5, with its runtime stack for libtorch, onnx, tvm and specialized accelerators. You like torchvision's retinanet? You like yolov5? You love yolort!

CNStream is a streaming framework for building Cambricon machine learning pipelines http://forum.cambricon.com https://gitee.com/SolutionSDK/CNStream

DELTA is a deep learning based natural language and speech processing platform.

Inference with state-of-the-art models (pre-trained by LD-Net / AutoNER / VanillaNER / ...)

Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.

MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.

Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.

A simple implementation of the OWL2 RL Profile on top of RDFLib: it expands the graph with all possible triples that OWL RL defines. It can be used together with RDFLib to expand an RDFLib Graph object, or as a stand alone service with its own serialization.

Nostril: Nonsense String Evaluator

A curated list of research in machine learning system. I also summarize some papers if I think they are really interesting.

LoMRF is an open-source implementation of Markov Logic Networks

Deploy a ML inference service on a budget in less than 10 lines of code.

The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform

Openllet is an OWL 2 reasoner in Java, build on top of Pellet.

Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.

An implementation of MTCNN Face detector using OpenCV's DNN module

go binding for mxnet c_predict_api to do inference with pre-trained model

Ordinal regression in Python

Serve your Rubix ML models in production with scalable stand-alone model inference servers.

This is a repository for an image classification inference API using the Gluoncv framework. The inference REST API works on CPU/GPU. It's supported on Windows and Linux Operating systems. Models trained using our Gluoncv Classification training repository can be deployed in this API. Several models can be loaded and used at the same time.

🎨 A complete Pattern Matching library for TypeScript, with smart type inference.