Top 162 inference open source projects

A uniform interface to run deep learning models from multiple frameworks

Sample code for the NIPS paper "Scalable Variational Inference for Dynamical Systems"

Pytorch-Named-Entity-Recognition-with-BERT

a fast and user-friendly runtime for transformer inference (Bert, Albert, GPT2, Decoders, etc) on CPU and GPU.

High-efficiency floating-point neural network inference operators for mobile, server, and Web

Multi Model Server is a tool for serving neural net models for inference

The challenge projects for Inferencing machine learning models on iOS

💎1MB lightweight face detection model (1MB轻量级人脸检测模型)

TensorFlow models accelerated with NVIDIA TensorRT

Lua Language Server coded by Lua

Embedded and mobile deep learning research resources

Shape and dimension inference (Keras-like) for PyTorch layers and neural networks

LightSeq: A High Performance Inference Library for Sequence Processing and Generation

Runtime type system for IO decoding/encoding

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.

A scalable inference server for models optimized with OpenVINO™

TensorFlow examples in C, C++, Go and Python without bazel but with cmake and FindTensorFlow.cmake

A REST API for Caffe using Docker and Go

Fast implementation of BERT inference directly on NVIDIA (CUDA, CUBLAS) and Intel MKL

Train a state-of-the-art yolov3 object detector from scratch!

Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection, NeurIPS2020

Run TensorFlow models in C++ without installation and without Bazel

Snips NLU rust implementation

AI 尬舞机

Blind WAF identification tool

Small, dependency-free, fast Python package to infer binary file types checking the magic numbers signature

This is a repository for an object detection inference API using the Tensorflow framework.

convert mmdetection model to tensorrt, support fp16, int8, batch input, dynamic shape etc.

OpenNARS for Research 3.0+

HLS based Deep Neural Network Accelerator Library for Xilinx Ultrascale+ MPSoCs

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and …

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.

Documentation, models and code relating to the 3rd edition of the textbook Stochastic Modelling for Systems Biology

TypeQL: the query language of TypeDB - a strongly-typed database

An ICE-based predicate synthesizer for Horn clauses.

Serve scikit-learn, XGBoost, TensorFlow, and PyTorch models with AWS Lambda container images support.

Deep learning based gaze estimation demo with a fun feature :-)

Tensorflow C++ examples for Visual Studio. Features Pose Estimation and various techniques to utilize the Tensorflow C++ interface

Python package for the creation, manipulation, and learning of Causal DAGs

Detect various irregularities of a product as it moves along a conveyor belt.

A scalable, high-performance serving system for federated learning models

Ordinal Common-sense Inference

Exposes a serialized machine learning model through a HTTP API.

The Triton backend for the ONNX Runtime.

This fork of BVLC/Caffe is dedicated to supporting Cambricon deep learning processor and improving performance of this deep learning framework when running on Machine Learning Unit(MLU).

Chainer implementation of Dense Fusion

Noisy network measurement with stan

A Unified Framework for Random Forest Prediction Error Estimation

C++ Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, OpenVINO, ncnn, MNN, SNPE, Arm NN, NNabla, ONNX Runtime, LibTorch, TensorFlow

Pyinfer is a model agnostic tool for ML developers and researchers to benchmark the inference statistics for machine learning models or functions.

A scalable python-based framework for gene regulatory network inference using tree-based ensemble regressors.

🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

Task-based gene regulatory network inference using single-cell or bulk gene expression data conditioned on a prior network.

⚡ Open-source software for deep learning-based digital pathology

This is a repository for a No-Code object detection inference API using the OpenVINO. It's supported on both Windows and Linux Operating systems.

Spark ML Lib serving library

This is a repository for a semantic segmentation inference API using the OpenVINO toolkit