Top 162 inference open source projects

Dawn Bench Entries
DAWNBench: An End-to-End Deep Learning Benchmark and Competition
Models
Model Zoo for Intel® Architecture: contains Intel optimizations for running deep learning workloads on Intel® Xeon® Scalable processors
Adlik
Adlik: Toolkit for Accelerating Deep Learning Inference
Bmw Yolov4 Inference Api Gpu
This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.
Tensorrt Laboratory
Explore the Capabilities of the TensorRT Platform
Libonnx
A lightweight, portable pure C99 onnx inference engine for embedded devices with hardware acceleration support.
Frozen Graph Tensorflow
Save, Load Frozen Graph and Run Inference From Frozen Graph in TensorFlow 1.x and 2.x
Dl inference
通用深度学习推理服务,可在生产环境中快速上线由TensorFlow、PyTorch、Caffe框架训练出的深度学习模型。
Elfi
ELFI - Engine for Likelihood-Free Inference
Volksdep
volksdep is an open-source toolbox for deploying and accelerating PyTorch, ONNX and TensorFlow models with TensorRT.
Pytorch Cpp Inference
Serving PyTorch 1.0 Models as a Web Server in C++
Pytorch Cpp
PyTorch C++ inference with LibTorch
Object Detection Api
Yolov3 Object Detection implemented as APIs, using TensorFlow and Flask
Ml privacy meter
Machine Learning Privacy Meter: A tool to quantify the privacy risks of machine learning models with respect to inference attacks, notably membership inference attacks
Server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Gpmp2
Gaussian Process Motion Planner 2
Bert Ner Tf
Named Entity Recognition with BERT using TensorFlow 2.0
Bmw Labeltool Lite
This repository provides you with a easy to use labeling tool for State-of-the-art Deep Learning training purposes.
Onnxt5
Summarization, translation, sentiment-analysis, text-generation and more at blazing speed using a T5 version implemented in ONNX.
Forward
A library for high performance deep learning inference on NVIDIA GPUs.
Json Autotype
Automatic Haskell type inference from JSON input
Compute Engine
Highly optimized inference engine for Binarized Neural Networks
Embedded Ai.bench
benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.
Ml Model Ci
MLModelCI is a complete MLOps platform for managing, converting, profiling, and deploying MLaaS (Machine Learning-as-a-Service), bridging the gap between current ML training and serving systems.
Tf2
An Open Source Deep Learning Inference Engine Based on FPGA
Yolov5 Rt Stack
Yet another yolov5, with its runtime stack for libtorch, onnx, tvm and specialized accelerators. You like torchvision's retinanet? You like yolov5? You love yolort!
Cnstream
CNStream is a streaming framework for building Cambricon machine learning pipelines http://forum.cambricon.com https://gitee.com/SolutionSDK/CNStream
Lightner
Inference with state-of-the-art models (pre-trained by LD-Net / AutoNER / VanillaNER / ...)
Sagemaker Inference Toolkit
Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
Mivisionx
MIVisionX toolkit is a set of comprehensive computer vision and machine intelligence libraries, utilities, and applications bundled into a single toolkit. AMD MIVisionX also delivers a highly optimized open-source implementation of the Khronos OpenVX™ and OpenVX™ Extensions.
Kglab
Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.
Owl Rl
A simple implementation of the OWL2 RL Profile on top of RDFLib: it expands the graph with all possible triples that OWL RL defines. It can be used together with RDFLib to expand an RDFLib Graph object, or as a stand alone service with its own serialization.
Nostril
Nostril: Nonsense String Evaluator
Awesome System For Machine Learning
A curated list of research in machine learning system. I also summarize some papers if I think they are really interesting.
Lomrf
LoMRF is an open-source implementation of Markov Logic Networks
Budgetml
Deploy a ML inference service on a budget in less than 10 lines of code.
Ncnn Benchmark
The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform
Openllet
Openllet is an OWL 2 reasoner in Java, build on top of Pellet.
People Counter Python
Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.
Opencv Mtcnn
An implementation of MTCNN Face detector using OpenCV's DNN module
Go Mxnet Predictor
go binding for mxnet c_predict_api to do inference with pre-trained model
Bevel
Ordinal regression in Python
Server
Serve your Rubix ML models in production with scalable stand-alone model inference servers.
Bmw Classification Inference Gpu Cpu
This is a repository for an image classification inference API using the Gluoncv framework. The inference REST API works on CPU/GPU. It's supported on Windows and Linux Operating systems. Models trained using our Gluoncv Classification training repository can be deployed in this API. Several models can be loaded and used at the same time.
Ts Pattern
🎨 A complete Pattern Matching library for TypeScript, with smart type inference.
1-60 of 162 inference projects