MobilePose-Pi
This is a deployment of MobilePose-pytorch for Raspberry Pi.
Structure
| Files | Explanation |
|---|---|
| cocoapi | this is from https://github.com/cocodataset/cocoapi |
| mobilepose-pi | execution directory for Raspberry Pi |
| models | pretrained models (.t7) are stored |
| pose_dataset | MPII dataset (csv files) |
| results | keypoints of each model's grountruth & prediction |
| OpenCV_Installation_Guide.md | OpenCV installation guide |
| PyTorch_Installation_Guide.md | PyTorch installation guide |
| README.md | general explanation about this repo |
| ShuffleNetV2.py | network architecture of ShufflenetV2 |
| Study_on_results.md | study on validation of our results |
| coco_utils.py | generating groundtruth&prediction json files |
| dataloader.py | multi-thread dataloader with augmentations |
| dataset_factory.py | get dataset's keypoint from csv files |
| estimator.py | this is for run_webcam.py |
| eval_pc.py | scripts to evaluate each model |
| mobilenetv2.py | network architecture of MobilenetV2 |
| networks.py | get model's path and input's height & width network architecture of Resnet |
| pycocotools | link to cocoapi/PythonAPI/pycocotools |
| requirements.txt | libraries needed to run scripts |
| run_webcam.py | real-time pose estimation using webcam |
| training.py | training models |
Note: MPII is used for training and COCO is used for evaluation.
Results
Note:As you can see, each model has room to be improved. Any ideas or opinions are welcome!
| Network | FPS | Size(MB) | mAP |
|---|---|---|---|
| 1.0 ShufflenetV2 | 1.09 | 5.3 | 0.000 |
| 1.0 MobilenetV2 | 0.66 | 9.3 | 0.045 |
| Resnet18 | 0.39 | 44.9 | 0.257 |
You can get more detailed info for this results here.
Requirements
For 'mobilepose-pi', you need following environment.
Raspberry Pi (We used Pi 2 Model B for this repo)
Rasbian stretch (32bit)
Python 2.7
Installation
Before installation, please change swap size.
At Pi terminal, open 'dphys-swapfile'.
sudo nano /etc/dphys-swapfileand change CONF_SWAPSIZE. (default is set to 100.)
CONF_SWAPSIZE=2048Activate it.
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile startNow you can install following things into your Pi.
PyTorch-0.2.0(499MB) - Follow my guide.
torchvision-0.2.1(2.7MB) - Please install it from source. Repo is here.
OpenCV-3.4.0(1.8GB) - Follow this guide.
Also, don't forget to install libraries.
pip install -r requirements.txtAt the end, change CONF_SWAPSIZE to 100 again.
Execution
You can evaluate your network on PC first.
If the result seems to be good, you can try it on Pi using 'mobilepose-pi'.
For mobilenet:
python eval_pc.py --model mobilenetFor resnet:
python eval_pc.py --model resnetFor shufflenet:
python eval_pc.py --model shufflenet*I just reused the same pretrained models MobilePose-pytorch author provided.
For shufflenet I used my original model.
Of course, you can use your own models also.
You can try real-time estimation using 'run_webcam.py'
(I didn't change this file from original repo.)
Please connect webcam to PC first. Then try this.
python run_webcam.py --model [name of model]Currently, run_webcam.py doesn't support shufflenet.
Training
You can train three models (shufflenet/mobilenet/resnet) on your PC.
You need to download MPII training Images(12.9GB). This is from here.
After extraction, please set ROOT_DIR at dataloader.py (line.197).
Different from MobilePose-pytorch(original repo), the command is
python train.py --model=[name_of_model_you_want_to_train] --retrain=[bool]
If you want to train shufflenet, you can do it just change model name.
(i.e. python train.py --model=shufflenet)
Model conversion
Note: development of this feature is currently suspended.
For conversion, I mainly adopted this tutorial.
I recommend to do this via conda.
1.You need to set up PC environment.
Citation
Original Repo:
https://github.com/YuliangXiu/MobilePose-pytorch
ShufflenetV2 paper:
https://arxiv.org/pdf/1807.11164.pdf
ShufflenetV2 implementation:
https://github.com/ericsun99/Shufflenet-v2-Pytorch
ONNX:
https://github.com/onnx/onnx
PyTorch installation:
https://wormtooth.com/20180617-pytorch-on-raspberrypi/
torchvision installation:
https://github.com/pytorch/vision
OpenCV installation:
https://www.pyimagesearch.com/2017/09/04/raspbian-stretch-install-opencv-3-python-on-your-raspberry-pi/