Top 162 inference open source projects

Dawn Bench Entries
DAWNBench: An End-to-End Deep Learning Benchmark and Competition
Model Zoo for Intel® Architecture: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors
Adlik: Toolkit for Accelerating Deep Learning Inference
Bmw Yolov4 Inference Api Gpu
This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.
Tensorrt Laboratory
Explore the Capabilities of the TensorRT Platform
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
Frozen Graph Tensorflow
Save, Load Frozen Graph and Run Inference From Frozen Graph in TensorFlow 1.x and 2.x
Dl inference
ELFI - Engine for Likelihood-Free Inference
volksdep is an open-source toolbox for deploying and accelerating PyTorch, ONNX and TensorFlow models with TensorRT.
Pytorch Cpp Inference
Serving PyTorch 1.0 Models as a Web Server in C++
Pytorch Cpp
PyTorch C++ inference with LibTorch
Object Detection Api
Yolov3 Object Detection implemented as APIs, using TensorFlow and Flask
Ml privacy meter
Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Gaussian Process Motion Planner 2
Bert Ner Tf
Named Entity Recognition with BERT using TensorFlow 2.0
Bmw Labeltool Lite
This repository provides you with a easy to use labeling tool for State-of-the-art Deep Learning training purposes.
Summarization, translation, sentiment-analysis, text-generation and more at blazing speed using a T5 version implemented in ONNX.
A library for high performance deep learning inference on NVIDIA GPUs.
Json Autotype
Automatic Haskell type inference from JSON input
Compute Engine
Highly optimized inference engine for Binarized Neural Networks
Embedded Ai.bench
benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.
Ml Model Ci
MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
An Open Source Deep Learning Inference Engine Based on FPGA
Yolov5 Rt Stack
Yet another yolov5, with its runtime stack for libtorch, onnx, tvm and specialized accelerators. You like torchvision's retinanet? You like yolov5? You love yolort!
CNStream is a streaming framework for building Cambricon machine learning pipelines
Inference with state-of-the-art models (pre-trained by LD-Net / AutoNER / VanillaNER / ...)
Sagemaker Inference Toolkit
Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.
Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.
Owl Rl
A simple implementation of the OWL2 RL Profile on top of RDFLib: it expands the graph with all possible triples that OWL RL defines. It can be used together with RDFLib to expand an RDFLib Graph object, or as a stand alone service with its own serialization.
Nostril: Nonsense String Evaluator
Awesome System For Machine Learning
A curated list of research in machine learning system. I also summarize some papers if I think they are really interesting.
LoMRF is an open-source implementation of Markov Logic Networks
Deploy a ML inference service on a budget in less than 10 lines of code.
Ncnn Benchmark
The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform
Openllet is an OWL 2 reasoner in Java, build on top of Pellet.
People Counter Python
Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.
Opencv Mtcnn
An implementation of MTCNN Face detector using OpenCV's DNN module
Go Mxnet Predictor
go binding for mxnet c_predict_api to do inference with pre-trained model
Ordinal regression in Python
Serve your Rubix ML models in production with scalable stand-alone model inference servers.
Bmw Classification Inference Gpu Cpu
This is a repository for an image classification inference API using the Gluoncv framework. The inference REST API works on CPU/GPU. It's supported on Windows and Linux Operating systems. Models trained using our Gluoncv Classification training repository can be deployed in this API. Several models can be loaded and used at the same time.
Ts Pattern
🎨 A complete Pattern Matching library for TypeScript, with smart type inference.
1-60 of 162 inference projects