BUG1989 / Ncnn Benchmark
The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform
Stars: ✭ 70
Projects that are alternatives of or similar to Ncnn Benchmark
Embedded Ai.bench
benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.
Stars: ✭ 131 (+87.14%)
Mutual labels: arm, inference, benchmark
BMW-IntelOpenVINO-Segmentation-Inference-API
This is a repository for a semantic segmentation inference API using the OpenVINO toolkit
Stars: ✭ 31 (-55.71%)
Mutual labels: inference, deeplearning
Tensorflow Cmake
TensorFlow examples in C, C++, Go and Python without bazel but with cmake and FindTensorFlow.cmake
Stars: ✭ 418 (+497.14%)
Mutual labels: cmake, inference
Metabench
A simple framework for compile-time benchmarks
Stars: ✭ 146 (+108.57%)
Mutual labels: cmake, benchmark
Arm Cmake Toolchains
CMake toolchain configurations for ARM
Stars: ✭ 148 (+111.43%)
Mutual labels: cmake, arm
InferenceHelper
C++ Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, OpenVINO, ncnn, MNN, SNPE, Arm NN, NNabla, ONNX Runtime, LibTorch, TensorFlow
Stars: ✭ 142 (+102.86%)
Mutual labels: inference, deeplearning
Dawn Bench Entries
DAWNBench: An End-to-End Deep Learning Benchmark and Competition
Stars: ✭ 254 (+262.86%)
Mutual labels: deeplearning, inference
Awesome Ml Demos With Ios
The challenge projects for Inferencing machine learning models on iOS
Stars: ✭ 741 (+958.57%)
Mutual labels: demo, inference
Aoe
AoE (AI on Edge，终端智能，边缘计算) 是一个终端侧AI集成运行时环境 (IRE)，帮助开发者提升效率。
Stars: ✭ 759 (+984.29%)
Mutual labels: demo, benchmark
Neuropod
A uniform interface to run deep learning models from multiple frameworks
Stars: ✭ 858 (+1125.71%)
Mutual labels: deeplearning, inference
Realtime Detectron
Real-time Detectron using webcam.
Stars: ✭ 42 (-40%)
Mutual labels: demo, deeplearning
Cartoonize
A demo webapp to convert images and videos into cartoon!
Stars: ✭ 215 (+207.14%)
Mutual labels: demo, deeplearning
Polymcu
An open framework for micro-controller software
Stars: ✭ 173 (+147.14%)
Mutual labels: cmake, arm
Pytorch Cpp
PyTorch C++ inference with LibTorch
Stars: ✭ 194 (+177.14%)
Mutual labels: demo, inference
BMW-IntelOpenVINO-Detection-Inference-API
This is a repository for a No-Code object detection inference API using the OpenVINO. It's supported on both Windows and Linux Operating systems.
Stars: ✭ 66 (-5.71%)
Mutual labels: inference, deeplearning
Bmw Tensorflow Inference Api Cpu
This is a repository for an object detection inference API using the Tensorflow framework.
Stars: ✭ 158 (+125.71%)
Mutual labels: deeplearning, inference
Bmw Yolov4 Inference Api Gpu
This is a repository for an nocode object detection inference API using the Yolov3 and Yolov4 Darknet framework.
Stars: ✭ 237 (+238.57%)
Mutual labels: deeplearning, inference
Ultra Light Fast Generic Face Detector 1mb
💎1MB lightweight face detection model (1MB轻量级人脸检测模型)
Stars: ✭ 6,182 (+8731.43%)
Mutual labels: arm, inference
Ros Academy For Beginners
中国大学MOOC《机器人操作系统入门》代码示例 ROS tutorial
Stars: ✭ 861 (+1130%)
Mutual labels: cmake, demo
ncnn-benchmark
The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform https://github.com/Tencent/ncnn
Hardware Platform
| Device | System | CPU-Family | CPU-Num | Freq |
|---|---|---|---|---|
| RK3288 | Android 5.1 | Cortex-A17 | 4 | 1.8GHz |
| Qualcomm820 | Android 6.0 | Kryo | 2+2 | 2.15GHz/1.6GHz |
| Mi5 | Android 7.1.2 | Kryo | 2+2 | 1.8GHz/1.3GHz |
| Hi3519 | Linux 3.18.20 | Cortex-A17 | 1 | 1.2GHz |
Runtime Environment
1.Using the cpu working in performance model.
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
2.Loop a hundred times to take the minimum value of Inference time-consuming.
Result
| Devices | Models | Input Size | Single-Thread(ms) | Multi-Threads(ms) |
|---|---|---|---|---|
| RK3288 | SqueezeNet v1.1 | 227x227x3 | 194 | 73 |
| MobileNet v1.0 | 224x224x3 | 324 | 115 | |
| ResNet18 | 224x224x3 | 759 | 250 | |
| ResNet50 | 224x224x3 | 1855 | 810 | |
| GoogleNet v1.0 | 224x224x3 | 701 | 231 | |
| VGG16 | 224x224x3 | 3449 | 1506 | |
| MobileNet-SSD | 300x300x3 | 652 | 245 | |
| Qualcomm820 | SqueezeNet v1.1 | 227x227x3 | 91 | 47 |
| MobileNet v1.0 | 224x224x3 | 150 | 70 | |
| ResNet18 | 224x224x3 | 355 | 152 | |
| ResNet50 | 224x224x3 | 735 | 309 | |
| GoogleNet v1.0 | 224x224x3 | 371 | 161 | |
| VGG16 | 224x224x3 | 1928 | 769 | |
| MobileNet-SSD | 300x300x3 | 330 | 160 | |
| Mi5 | SqueezeNet v1.1 | 227x227x3 | 98 | 51 |
| MobileNet v1.0 | 224x224x3 | 189 | 79 | |
| Hi3519 | SqueezeNet v1.1 | 227x227x3 | 397 | null |
| MobileNet v1.0 | 224x224x3 | 666 | null | |
| ResNet18 | 224x224x3 | 1608 | null | |
| GoogleNet v1.0 | 224x224x3 | 1410 | null |
User Guide
1. Build the benchmark demo
build demo for Linux-x86
./build.sh linux
build demo for Android
./build.sh android
2. How to run the executable files.
If you build demo for linux success,and want to run the benchmark demo.
$ cp ./models/classification/squeezenet.param ./build-linux/install/bin/
$ cp ./models/classification/squeezenet.bin ./build-linux/install/bin/
$ cd ./build-linux/install/bin/
$ ./ncnn_classify squeezenet.param squeezenet.bin 227 227 1 1
Demo running params:
./ncnn_classify <ncnn-param-file> <ncnn-model-bin-file> <input-width> <input-height> <loops-num> <threads-num>
Example:
[email protected]:~/ncnn-benchmark/build-linux/install/bin$ ./ncnn_classify squeezenet.param squeezenet.bin 227 227 10 2
--- NCNN Classification Benchmark Demo --- 22:41:09 Dec 26 2017
Loops : 10
Threads : 2
Time cost: Max 263.338 ms, Min 247.209 ms, Avg 252.652 ms.
The End
Thanks to ncnn's author nihui and all the contributors for sharing this framework.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].