Trending in 3D Vision
Deep learning not only provides good optimization techniques on feaure engineering, but also gives us unimaginable possibility to combine our intutions into research that could explore this 3D world better. In the research field of 3D Vision, a lot of excellent research work is happening. It's not enough to simply divide them into stereo vision, multi-view, monucular based 3D, we actually want to sort out research topics that could both connect with and distinguish from 2D vision. People have done paper collections on 3D as link1 link2, but they don't usually give us a good bird view, or most-updated infos. Based on above reasons, we create this special collection by summerizing the general ideas behind some most recent papers in 3D vision.
1. Single Image 3D & Unsupervised Monucular Video Depth
[Zhang et al. NIPS 2018] Learning to Reconstruct Shapes from Unseen Classes [Project] [Code]
[Wu et al. ECCV 2018] Learning Shape Priors for Single-View 3D Completion and Reconstruction [Project] [Code]
[Niu et al. CVPR 2018] Im2Struct: Recovering 3D Shape Structure from a Single RGB Image [Code]
[Zou et al. CVPR 2018] LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image [Code] [Video]
[Mahj. et al. CVPR 2018] Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints [Project] [Code]
[Yang et al. CVPR 2018] LEGO: Learning Edge with Geometry all at Once by Watching Videos [Demo] [Code]
[Zou et al. ECCV 2018] DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency [Project] [Code]
(Note: more papers on 'Monucular Video Depth' in CVPR 2017, 2018, ECCV 2018, along with the Robust Vision Challenge)
2. 3D Generative Model for Vision beyond Visiable
[[Eslami et al.](Science-Machine Learning)]Neural scene representation and rendering [Project] Code[tf-gqn][gqn-datasets] [Pytorch-qgn]'seems to run quite slow'
[Tulsiani et al. ECCV 2018] Layer-structured 3D Scene Inference via View Synthesis [Project] [Code]
[Rama. et al. ECCV 2018] Sidekick Policy Learning for Active Visual Exploration [Project] [Code]
[Song et al. CVPR 2018] Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View [Project] [Code]
3. Pose Estimation
3.1 Scene Layout and Object Pose
[Wang et al. ARXIV pre-print 2019]DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion [Code] [Project] [Video]
[Zhao et al. ARXIV pre-print 2019]Estimating 6D Pose From Localizing Designated Surface Keypoints [Code]
[Huang et al. NIPS 2018] Cooperative Holistic Scene Understanding: Unifying3D Object, Layout, and Camera Pose Estimation [Video]
[Trem. et al. CoRL 2018] Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects [Project] [Code]
[Sundermeyer et al. ECCV 2018]Implicit 3D Orientation Learning for 6D Object Detection from RGB Images(Best paper Award) [Code] [Supplement] [Video]
[Tulsiani et al. CVPR 2018] Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene [Project] [Code]
[Tulsiani et al. CVPR 2018]Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction [Project] [Code]
[Tekin et al. CVPR 2018] Real-Time Seamless Single Shot 6D Object Pose Prediction [Code] [Supp.]
[Li et. al. ECCV 2018] DeepIM: Deep Iterative Matching for 6D Pose Estimation [Code]
[Qi et al. CVPR 2018] Frustum PointNets for 3D Object Detection from RGB-D Data [Project] [Code]
3.2 Body Pose
[Guo et al. ECCV 2018] Neural Graph Matching Networks for Fewshot 3D Action Recognition (Pose for action recognition)
[Groueix et al. ECCV 2018] 3D-CODED : 3D Correspondences by Deep Deformation [Project] [Code]
[Joo et al. CVPR 2018] Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies [Project] [Supp.]
[Riza et al. CVPR 2018] Dense Human Pose Estimation In The Wild [Project] [Code]
[Pavl. et al. CVPR 2018] Learning to Estimate 3D Human Pose and Shape from a Single Color Image [Project]
3.3 Face Pose
[Moniz et al. NIPS 2018] Unsupervised Depth Estimation, 3D Face Rotation and Replacement [Code releasing soon]
[Feng et al. ECCV 2018] Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network [Code] [Video]
[Genova et al. CVPR 2018] Unsupervised Training for 3D Morphable Model Regression [Project] [Code]
[Deng et al. CVPR 2018] UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition
3.4 Hand Pose
[Tsoli et al. ECCV 2018] Joint 3D Tracking of a Deformable Object in Interaction with a Hand
[Ge et al. CVPR 2018] Hand PointNet: 3D Hand Pose Estimation using Point Sets [Project] [Code] [VIdeo]
[Baek et al. CVPR 2018] Augmented skeleton space transfer for depth-based hand pose estimation
(Note: more papers on 'Hand Pose Estimation' in CVPR 2018, ECCV 2018)
4. Disentangle Representations in 3D
[Yao et al. NIPS 2018] 3D-Aware Scene Manipulation via Inverse Graphics [Project]
[Smith et al. NIPS 2018] Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation [Project]
[Zhu et al. NIPS 2018] Visual Object Networks: Image Generation with Disentangled 3D Representation
5. Unsupervised Key Points Detection
[Suwa. et al. NIPS 2018] Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning [Project] [Code]
[Zhou et al. ECCV 2018] Unsupervised Domain Adaptation for 3D Keypoint Estimation via View Consistency [Code] [Results]
6. Point Cloud(PCL) Processing
6.1 Neural Network for 3D data
[Tata. et al. CVPR 2018] Tangent Convolutions for Dense Prediction in 3D [Code]
[Su et al. CVPR 2018] SPLATNet: Sparse Lattice Networks for Point Cloud Processing [Project] [Code]
[Qi et al. NIPS 2017] PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space [Project] [Code]
[Weiler et al. NIPS 2018] 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data [Project] [Code]
[Sung et al. NIPS 2018] Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions [Project] [Code]
6.2 3D Registration & Rendering
[Nguy. et al. NIPS 2018] RenderNet: A deep convolutional network for differentiable rendering from 3D shapes
[Kim et al. ECCV 2018] Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures [Project] [Video]
[Sung et al. Siggraph Asia 2017] ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling [Project] [Code]
[Zhu et al. SIGGRAPH Asia 2018] SCORES: Shape Composition with Recursive Substructure Priors [Project] [Code]
[Este. et al. 3DV 2018] Robust Fitting of Subdivision Surfaces for Smooth Shape Analysis [Project] [Code]
[Kato et al. CVPR 2018] Neural 3D Mesh Renderer [Project] [Code]
7. SLAM today
7.1 3D Reconstruction & SLAM
[Shi et al. ECCV 2018] PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D Reconstruction [Project] [Code]
[Bloesch et al. CVPR 2018] CodeSLAM — Learning a Compact, Optimisable Representation for Dense Visual SLAM [Project] [Video]
[Berg. et al. ICRA 2018] Online Photometric Calibration of Auto Exposure Video for Realtime Visual Odometry and SLAM [Project] [Code]
7.2 Object-aware SLAM
[Rünz et al. ISMAR 2018] MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects [Project] [Code] [Demo]
[McCo. et al. 3DV 2018] Fusion++: Volumetric Object-Level SLAM [Video]
[Zhou et al. ECCV 2018] DeepTAM: Deep Tracking and Mapping [Project] [Code]
7.3 3D Photography
[Hedman et al. SIGGRAPH] Instant 3D Photography [Project] [Code]
[Chen et al. ECCV 2018] PS-FCN: A Flexible Learning Framework for Photometric Stereo [Project] [Code]
Misc:
Workshops
Dataset 2018
[Trem. et al. CVPR 2018 Workshop] Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation [Dataset]
[Sun et al. CVPR 2018] Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling [Dataset]
[Muel. et al. CVPR 2018] GANerated Hands Dataset [Dataset]
SUMO challenge dataset [Dataset] [Code]
More resources
Amesome 3D machine learning collection here