AI-performance / Embedded Ai.bench
Licence: other
benchmark for embededded-ai deep learning inference engines, such as NCNN / TNN / MNN / TensorFlow Lite etc.
Stars: ✭ 131
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Embedded Ai.bench
Ncnn Benchmark
The benchmark of ncnn that is a high-performance neural network inference framework optimized for the mobile platform
Stars: ✭ 70 (-46.56%)
Mutual labels: arm, inference, benchmark
Adversarial Robustness Toolbox
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
Stars: ✭ 2,638 (+1913.74%)
Mutual labels: artificial-intelligence, inference
Ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
Stars: ✭ 13,376 (+10110.69%)
Mutual labels: artificial-intelligence, inference
Caffe Hrt
Heterogeneous Run Time version of Caffe. Added heterogeneous capabilities to the Caffe, uses heterogeneous computing infrastructure framework to speed up Deep Learning on Arm-based heterogeneous embedded platform. It also retains all the features of the original Caffe architecture which users deploy their applications seamlessly.
Stars: ✭ 271 (+106.87%)
Mutual labels: artificial-intelligence, arm
Tengine
Tengine is a lite, high performance, modular inference engine for embedded device
Stars: ✭ 4,012 (+2962.6%)
Mutual labels: artificial-intelligence, arm
Ultra Light Fast Generic Face Detector 1mb
💎1MB lightweight face detection model (1MB轻量级人脸检测模型)
Stars: ✭ 6,182 (+4619.08%)
Mutual labels: arm, inference
Stock Prediction
Smart Algorithms to predict buying and selling of stocks on the basis of Mutual Funds Analysis, Stock Trends Analysis and Prediction, Portfolio Risk Factor, Stock and Finance Market News Sentiment Analysis and Selling profit ratio. Project developed as a part of NSE-FutureTech-Hackathon 2018, Mumbai. Team : Semicolon
Stars: ✭ 125 (-4.58%)
Mutual labels: artificial-intelligence
Perceiver Pytorch
Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch
Stars: ✭ 130 (-0.76%)
Mutual labels: artificial-intelligence
P2plab
performance benchmark infrastructure for IPLD DAGs
Stars: ✭ 126 (-3.82%)
Mutual labels: benchmark
Meter
Meter - is a simple micro-benchmarking tool for Android (and Java) projects. This is not a profiler, this is very small utility class that designed for making benchmarking easy. Nothing more. Alternative to Android Jetpack Benchmark.
Stars: ✭ 125 (-4.58%)
Mutual labels: benchmark
Mixbench
A GPU benchmark tool for evaluating GPUs on mixed operational intensity kernels (CUDA, OpenCL, HIP, SYCL)
Stars: ✭ 130 (-0.76%)
Mutual labels: benchmark
Leagueai
LeagueAI software framework for League of Legends that provides information about the state of the game based on Image Recognition using OpenCV and Pytorch.
Stars: ✭ 128 (-2.29%)
Mutual labels: artificial-intelligence
Actors
Evaluation of API and performance of different actor libraries
Stars: ✭ 125 (-4.58%)
Mutual labels: benchmark
Ai Learning Roadmap
List of all AI related learning materials and practical tools to get started with AI apps
Stars: ✭ 130 (-0.76%)
Mutual labels: artificial-intelligence
Robovision
AI and machine leaning-based computer vision for a robot
Stars: ✭ 126 (-3.82%)
Mutual labels: artificial-intelligence
Xqwlight
XiangQi Wizard Light - a Simple but Strong XiangQi (Chinese Chess) AI Algorithm written in C++, Java, JavaScript and ActionScript
Stars: ✭ 130 (-0.76%)
Mutual labels: artificial-intelligence
Nas Benchmark
"NAS evaluation is frustratingly hard", ICLR2020
Stars: ✭ 126 (-3.82%)
Mutual labels: benchmark
Phantomjs On Raspberry
Phantomjs for raspberry pi (armv6/armv7) and aarch64/arm64
Stars: ✭ 128 (-2.29%)
Mutual labels: arm
Persephone
A tool for automatic phoneme transcription
Stars: ✭ 130 (-0.76%)
Mutual labels: artificial-intelligence
最新benchmark结果参见:https://github.com/AI-performance/embedded-ai.bench/releases
embedded-ai.bench
- 一键编译:拉取框架代码,编译库;
- 一键转换:拉取原始模型、编译转换工具、转换模型;
- 一键测速:拉取框架模型,测速。拉取框架模型,框架模型存放于不同的代码仓库中,执行测速过程会自动完成拉取。
跑起来
# tnn
cd ./tnn
./build_tnn_android.sh # follow & read `./tnn/build_tnn_android.sh` if build failed
cd -
# mnn
cd ./mnn
./build_mnn_android.sh # follow & read `./mnn/build_mnn_android.sh` if build failed
cd -
# ncnn
cd ./ncnn
./build_ncnn_android.sh # follow & read `./ncnn/build_ncnn_android.sh` if build failed
cd -
# tflite
cd ./tflite
./build_tflite_android.sh # follow & read `./tflite/build_tflite_android.sh` if build failed
cd -
# bench
python bench.py
# if execution is okay:
# ===> edit ./core/global_config.py
# ===> edit value of `GPU_REPEATS=1000`, `CPU_REPEATS=100`, `WARMUP=20`, `ENABLE_MULTI_THREADS_BENCH=True`
# ===> ./clean_bench_result.sh
# ===> python bench.py
# see benchmark result below
# ./tnn/*.csv
# ./mnn/*.csv
# ./ncnn/*.csv
目前支持模型有限,见:tnn-models、mnn-models、ncnn-models、tflite-models。
bench结果示例
项目架构
下图描述了本项目的架构:
- class Engine: 由
engine_config
创建每个框架的实例、加载模型models、编译引擎lib(TODO:待集成到py脚本),跑出bench结果; - BenchSum(TODO): 汇总各Engine实例的Bench结果;
- common model repo(TODO): 存放公共原始模型的仓库。如Caffe、TensorFlow框架的经典模型(MobileNetV1/V2等)。
- 各引擎独立模型仓库,如tnn-models。包含从公共公共模型仓库转换得到的tnn的模型、模型转换器的一键编译脚本(TODO)、模型转换的一键转换脚本(TODO)、模型版本一键刷新到README的脚本;
- 各引擎独立模型仓库独立定期更新(TODO);
- 每个Engine实例在执行过程中会拉取各自独立模型仓库的模型,为bench做准备。
LICENSE
禁止除AI-Performance开源组织以外的主体,【公开】发布【基于本项目的benchmark结果】,若公开发布则视为侵权,AI-Performance有权追诉法律责任。
AI-Performance开源组织,以中立、公平、公正、公开为组织准则,致力于打造制定AI领域的benchmark标准。
常见问题
1. adb找不到设备
一般是没开启「开发者模式」,在确认开启后,还是找不到,按照以下顺序一般都能解决:
- USB连接设置从「仅充电」改为「传输文件」;
- 换USB接口(可能电压);
- 加vendor ID到~/.android/adb_usb.ini然后adb kill-server再adb start-server;
- 重启手机;
- 重启电脑;
- 换数据线(我之前发现有这个情况);
- 华为手机的CD 驱动器Hisuite会占用adb。弹出该驱动器;
- 手机刷机。
开发须知
首次提交代码需执行以下命令,安装钩子。安装成功后,每次执行git commit
后会自动检查.pre-commit-config.yaml
里设定的检查项,如目前是针对Python代码做格式检查。
# 第一次执行钩子可能会比较慢
pre-commit install
# 若找不到则需要先安装pre-commit
pip install pre-commit
# 如需卸载则执行
pre-commit uninstall
如若找不到python3.8,可以安装minconda3,使用如下命令:
# 自动安装miniconda3,并写入当前用户的环境变量
.github/workflows/pre-commit-job.sh
# 提交代码前先创建一个名为dev_env_py的环境为例,遇到选择y
conda create -n dev_env_py python=3.8
# 激活刚创建的环境
conda activate dev_env_py
# 重新安装pre-commit
pre-commit install
若CI挂掉,查看Github Action的具体日志,是否由于timeout如
git clone
仓库,这时可以Re-run this jobs
重新运行。
单元测试
.github/workflows/unit-test-job.sh
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].