SSVIO
Graduation Project: A point cloud semantic segmentation and VIO based 3D reconstruction method using RGB-D and IMU
Newest result
REMEMBER!
Note that the metric of pointcloud is same as the LSB of depth sensor!
The coordinate of camera and world is as follow(Red:X, Blue:Y, Green:Z):
Requirement
- gcc and cmake
- OpenCV:https://opencv.org/
- PCL:https://pointclouds.org/
- g2o:https://github.com/RainerKuemmerle/g2o
- Sophus:https://github.com/strasdat/Sophus
- Eigen:http://eigen.tuxfamily.org/index.php?title=Main_Page
- Pangolin:https://github.com/stevenlovegrove/Pangolin
- suitesparse:
apt install libsuitesparse-dev
- openni:
apt install libopenni2-dev
Usage
- Go to {YOUR_DIRECTORY}/SSVIO
- Run following code
mkdir build
cd ./build
cmake ..
make
- After generating bin file, go to {YOUR_DIRECTORY}/SSVIO/bin
- Run following code
Visual Reconstruction
- visual reconstruction app:
./run_visual_reconstruct
map pointcloud data will be saved at {YOUR_DIRECTORY}/SSVIO/savings/map.pcd
Saving Data
- photo shooting and data saving app:
./run_saving_data
Press "s" to save image and pointcloud at {YOUR_DIRECTORY}/SSVIO/savings
- Run at {YOUR_DIRECTORY}
bash ./removedata.sh
to remove all the saved files
Feature matching Pose optimization and Frame jointment
- feature matching app:
./run_feature_match
input saved data num * 2
output matched feature points
- pose optimization:
./run_g2o_optim
input saved data num * 2
output pose and matched feature points
- frame jointment:
./run_frame_jointment
input saved data num * 2
output frame_joint.pcd saved at {YOUR_DIRECTORY}/SSVIO/savings/pointcloud
Visualization
After running ./run_visual_reconstruct, you will see three windows:
- RGB for RGB image
- Depth for depth image
- PointCloud for PointCloud viewer
Press "Esc" at RGB window or press "Ctrl-C" at terminal to stop the program
Bug
- Use solvepnpRANSAC to filter outliers
- Jacobian of IMU measure
- Local g2o optimization error
- Not orthogonal error
Author
Student from HITSZ Automatic Control NRS-lab