Mazhichaoruya / Object Detection And Location Realsensed435
Programming Languages
Projects that are alternatives of or similar to Object Detection And Location Realsensed435
Object-Detection-and-location-RealsenseD435
I just Update the C++ Version this time,Python Version will be updated when I have free time.maybe next week.
ROS version ROS-Object-Detection-2Dto3D-RealsenseD435
Object Dection with SLAM ,With SLAM(SC-Lego-LOAM) :Perception-of-Autonomous-mobile-robot
Requirements
DNN Version
Ubuntu18.04 OR 16.04
Opencv 4.x
C++ 11_std At least,I used the C++ 17 std
Eigen3 :in absolutely Path /usr/local/eigen3
Cmake>= 3.17
PCL lib>=1.7.1
Intel Realsense SDK >=2.0
Yolov3/v4 by Darknet
Dlib #I had pushed on this Github repository on :Object-Detection-and-location-RealsenseD435/C++/dlib.zip,uzip it on this Path.
You can DownLoad the lastest version too.
The config weight and cfg is Yolov4 based EfficentNetB0 ,it is a smaller model than yolov4_tiny but the result better.it can be run over 10FPS on CPU.
Python Version
pyrealsense2.x
Opencv-python
numpy
with same as C++ version,the SDK of realsense D435 must have been installed
TenorRT version
A Nvidia GPU is essential.I used Yolov5 by pytorch. tensorrtx Yolov5
How to use
DNN
git clone https://github.com/Mazhichaoruya/Object-Detection-and-location-RealsenseD435.git
cd Object-Detection-and-location-RealsenseD435/DNN/engine/
wget https://pjreddie.com/media/files/yolov3.weights ;wget https://pjreddie.com/media/files/yolov3-tiny.weights
cd Object-Detection-and-location-RealsenseD435/;uzip dlib.zip
mv dlib /DNN
you Can change the engine path in src/main.cpp
on line 25-27
String yolo_tiny_model ="../engine/yolov3.weights";
String yolo_tiny_cfg = "../engine/yolov3.cfg";
String classname_path="../engine/coco.names";
You can use your weight by Darknet or others supported by DNN too
cd ..
mkdir bulid; cd build
cmake ..
make
./DNN_Yolo
Attention:Default parameter on line 251 and 252 in src/main.cpp
net.setPreferableBackend(DNN_BACKEND_OPENCV);// DNN_BACKEND_INFERENCE_ENGINE DNN_BACKEND_CUDA
net.setPreferableTarget(DNN_TARGET_CPU);//DNN_TARGET_CUDA
if you have IntelCore CPU you can chose "DNN_BACKEND_INFERENCE_ENGINE"to accelerate youe model--Openvino;
But you should make sure your CPU is Intel and the Contrib of Opencv has been installed.
If you have GPU(From Nvidia),You can Think about The Cuda acceleration.Before this you should reinstall Your Opencv(Version Most>4.2) with This:OpenCV_DNN
Open the Cuda Setting when CMake.
Python
When you install the essential job ,just run the main.py,the model will start to work.
cd Python
python3 main.py
TensorRT
At first,Go to tensorrtxget the tensorrt engine(Before this you must had installed Ubuntu,CUDA,TensorRT).
Then move the .enigine files to Yolov5-TensorRT-AGX_Xavier/engine
#define VIDEO_TYPE (0) //0 means laptop camera ;1 means images,2 means Videos,3 means RealsenseD435;
#define NET x // s m l x
You can change The Define type as you like，From src/main.cpp.
cd TensorRT;
mkdir bulid; cd build
cmake ..
make -j12
./Yolov5_trt
Example
DNN version on 9-21:
RGBD and Center position:
Point Cloud of Objects:
Yolov5 by TensorRT：
Next
The ROS and SLAM version had been uploaded,But it is just a Beginning,I will do more jod form my Graduation Project，if you were Interested in this, welcome to follow me！
深度相机目标检测DNN&TensorRT
DNN版本
请参考英文注释操作编译，默认使用CPU模型，Opencv 4.x，当使用给予EfficentNetB0轻量级网络的Yolov4时，在CPU上可以超过10FPS。实测AMD R5 4600H 15FPS。
TensorRT版本
仅支持N卡独显，C若使用CPU可以使用DNN模型，但是很难达到实时。
相关工作
tensorrtx
Yolov5
请参考如上工程的说明配置Cuda、TensorRT环境
使用
首先按照 tensorrtx获得Tensorrt可使用的模型，(前提是已经安装了 Ubuntu,CUDA,TensorRT，Opencv等必要的库).
移动 .enigine files 到 Yolov5-TensorRT-AGX_Xavier/engine目录下
#define VIDEO_TYPE (0) //0 means laptop camera ;1 means images,2 means Videos,3 means RealsenseD435;
#define NET x // s m l x
可以自行修改main.cpp的参数,NET表示s、m、l、x的模型类别，VIDEO_TYPE 表示输入数据类型，0-笔记本自带0号摄像头，1-图片，2-输入视频文件，3-深度相机（realsenseD435)
cd TensorRT;
mkdir bulid; cd build
cmake ..
make -j12
./Yolov5_trt
实例视频