Tengine Convert Tools
Introduction
Tengine Convert Tool supports converting multi framworks' models into tmfile that suitable for Tengine-Lite AI framework. Since this tool relys on protobuf to resolve proto file of Caffe, ONNX, TensorFlow, TFLite and so on, it can only run under x86 Linux system.
Install dependent libraries
- For loading caffe model or TensorFlow model.
sudo apt install libprotobuf-dev protobuf-compiler- If use the Fedora/CentOS ,use follow command instead.
sudo dnf install protobuf-devel
sudo dnf install boost-devel glog-develBuild Convert Tool
mkdir build && cd build
cmake ..
make -j`nproc` && make installExection File
- The exection should be under
./build/install/bin/named asconvert_tool.
Run Convert Tool
How to use
$ ./convert_tool -h
[Convert Tools Info]: optional arguments:
-h help show this help message and exit
-f input type path to input float32 tmfile
-p input structure path to the network structure of input model(*.prototxt, *.symbol, *.cfg)
-m input params path to the network params of input model(*.caffemodel, *.params, *.weight, *.pb, *.onnx, *.tflite)
-o output model path to output fp32 tmfile
To run the convert tool, running as following command, Note: The command examples are based on mobilenet model:
- Caffe
./install/bin/convert_tool -f caffe -p mobilenet_deploy.prototxt -m mobilenet.caffemodel -o mobilenet.tmfile- MxNet
./install/bin/convert_tool -f mxnet -p mobilenet1_0-symbol.json -m mobilene1_0-0000.params -o mobileent.tmfile- ONNX
./install/bin/convert_tool -f onnx -m mobilenet.onnx -o mobilenet.tmfile- TensorFlow
./install/bin/convert_tool -f tensorflow -m mobielenet_v1_1.0_224_frozen.pb -o mobilenet.tmfile- TFLITE
./install/bin/convert_tool -f tflite -m mobielenet.tflite -o mobilenet.tmfile- DarkNet: darknet only support for yolov3 model
./install/bin/convert_tool -f darknet -p yolov3.cfg -m yolov3.weights -o yolov3.tmfile- NCNN
./install/bin/convert_tool -f ncnn -p mobilenet.param -m mobilenet.bin -o mobilenet.tmfile- MegEngine
./install/bin/convert_tool -f megengine -m mobilenet.pkl -o mobilenet.tmfile- PaddlePaddle
./install/bin/convert_tool -f paddle -p inference.pdmodel -m inference.pdiparams -o mobilenetv2_paddle.tmfileHow to enable MegEngine support[optional]
- First of all, build MegEngine with DEBUG mode:
# clone MegEngine
git clone https://github.com/MegEngine/MegEngine.git
# prepare for building
cd MegEngine
./third_party/prepare.sh
./third_party/install-mkl.sh
mkdir build && cd build
# build MegEngine with DEBUG mode
cmake .. -DMGE_WITH_TEST=OFF -DMGE_BUILD_SDK=OFF -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX={PREDEFINED_INSTALL_PATH}
make -j`nproc`
make install
make develop
# export environment
export PYTHONPATH=/path/to/MegEngine/python_module
# test with python
python3 -c "import megengine"- Build Tengine convert tool
# clone Tengine convert tool
git clone https://github.com/OAID/Tengine-Convert-Tools
# build with MegEngine support
cmake -DBUILD_MEGENGINE_SERIALIZER=ON -DMEGENGINE_INSTALL_PATH={PREDEFINED_INSTALL_PATH} ..
make -j`nproc`
make install- Serialize your MegEngine model
# get a pre-trained resnet18 model from MegEngine Model Hub
import megengine.hub
resnet18 = megengine.hub.load("megengine/models", "resnet18", pretrained=True)
# use MegEngine trace to deal with downloaded model
from megengine.jit import trace
import megengine.functional as F
@trace(symbolic=True)
def pred_fun(data, *, net):
net.eval()
pred = net(data)
# if model has softmax
pred_normalized = F.softmax(pred)
return pred_normalized
# fill a random input for model
import numpy as np
data = np.random.random((1, 3, 224, 224)).astype(np.float32)
# trace and save the model
pred_fun.trace(data, net=resnet18)
pred_fun.dump('new_model.pkl')A jupyter notebook was offered for users, check MegEngine.ipynb.
- Convert MegEngine .pkl model to Tengine .tmfile
./install/bin/convert_tool -f megengine -m new_model.pkl -o resnet18.tmfileHow to add self define operator
- Adding operator's name defined file under operator/include directory that likes xxx.hpp and xxx_param.hpp (including operator's params);
- Adding operator's memory allocation (calculate the memory) under operator/operator directory;
- Register operator in project operators' registery under operator/operator/plugin/init.cpp file;
- After adding operator definition, you need to add operator into model serializers, these files are under tools directory. There are multiply framework model serializers, finding file name and .cpp file under that corresponding framwork folder. Following the other operator's definition in that .cpp file.
License
Tech Forum
- Github issues
- QQ groupchat: 829565581
- Email: [email protected]
- Tengine Community: http://www.tengine.org.cn/