All Projects → rludlow → 3d-pose-2d-keypoints

rludlow / 3d-pose-2d-keypoints

Licence: other
3d Human Pose Estimation from 2d Keypoints

Programming Languages

Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Projects that are alternatives of or similar to 3d-pose-2d-keypoints

LEMO
Official Pytorch implementation for 2021 ICCV paper "Learning Motion Priors for 4D Human Body Capture in 3D Scenes" and trained models / data
Stars: ✭ 149 (+166.07%)
Mutual labels:  motion-capture, pose-estimation
UniFormer
[ICLR2022] official implementation of UniFormer
Stars: ✭ 574 (+925%)
Mutual labels:  pose-estimation
PeekingDuck
A modular framework built to simplify Computer Vision inference workloads.
Stars: ✭ 143 (+155.36%)
Mutual labels:  pose-estimation
NSRMhand
[WACV 2020] "Nonparametric Structure Regularization Machine for 2D Hand Pose Estimation"
Stars: ✭ 95 (+69.64%)
Mutual labels:  pose-estimation
MobilePose-Pi
MobilePose deployment for Raspberry Pi
Stars: ✭ 15 (-73.21%)
Mutual labels:  pose-estimation
virtual drawing board
Virtual whiteboard with hand pose estimation
Stars: ✭ 30 (-46.43%)
Mutual labels:  pose-estimation
tf-cpn
Cascade Pyramid Netwrok
Stars: ✭ 22 (-60.71%)
Mutual labels:  pose-estimation
awesome-visual-localization-papers
The relocalization task aims to estimate the 6-DoF pose of a novel (unseen) frame in the coordinate system given by the prior model of the world.
Stars: ✭ 60 (+7.14%)
Mutual labels:  pose-estimation
flydra
live, low-latency markerless multi-camera 3D animal tracking system
Stars: ✭ 52 (-7.14%)
Mutual labels:  pose-estimation
OpenISS
OpenISS -- a unified multimodal motion data delivery framework.
Stars: ✭ 22 (-60.71%)
Mutual labels:  motion-capture
trt pose hand
Real-time hand pose estimation and gesture classification using TensorRT
Stars: ✭ 137 (+144.64%)
Mutual labels:  pose-estimation
blender-retarget
Applies animation from one armature to another
Stars: ✭ 137 (+144.64%)
Mutual labels:  motion-capture
qml-ar
Seamless Augmented Reality module for QML using UchiyaMarkers
Stars: ✭ 32 (-42.86%)
Mutual labels:  pose-estimation
icra20-hand-object-pose
[ICRA 2020] Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands
Stars: ✭ 42 (-25%)
Mutual labels:  pose-estimation
WebCamCap
Motion capture tool for 2D/3D motion capture with LED markers.
Stars: ✭ 20 (-64.29%)
Mutual labels:  motion-capture
realant
RealAnt robot platform for low-cost, real-world reinforcement learning
Stars: ✭ 40 (-28.57%)
Mutual labels:  pose-estimation
CenterNet-pytorch-lightning
Refactored implementation of CenterNet (Objects as Points - Zhou, Xingyi et. al.) shipping with PyTorch Lightning modules
Stars: ✭ 53 (-5.36%)
Mutual labels:  pose-estimation
FASSEG-repository
Datasets for multi-class and multi-pose face segmentation
Stars: ✭ 66 (+17.86%)
Mutual labels:  pose-estimation
OffsetGuided
Code for "Greedy Offset-Guided Keypoint Grouping for Human Pose Estimation"
Stars: ✭ 31 (-44.64%)
Mutual labels:  pose-estimation
realsense explorer bot
Autonomous ground exploration mobile robot which has 3-DOF manipulator with Intel Realsense D435i mounted on a Tracked skid-steer drive mobile robot. The robot is capable of mapping spaces, exploration through RRT, SLAM and 3D pose estimation of objects around it. This is an custom robot with self built URDF model.The Robot uses ROS's navigation…
Stars: ✭ 61 (+8.93%)
Mutual labels:  pose-estimation

3d Human Pose Estimation from 2d Keypoints

Overview

While humans can generally estimate with ease the 3d pose of a human in a 2d image, 3d pose estimation remains a challenging problem for machines. This project improves an algorithm that estimates 3d keypoints of human poses with 2d keypoints as the only input.

I pursue three key interventions to improve reconstruction acurracy over the entire dataset as well as a subset of poses with particularly high error in the benchmark model: a) modify the data normalization technique in preprocessing, b) modify the neural network architecture from a simple densely connected network to a multi-stage network modeled after state-of-the-art 2d pose estimation models, and c) generate synthetic data to augment training sets.

The interventions successfully reduced reconstruction error over the entire testing set (from Carnegie Mellon's Motion Capture Database) by 40%, and that of targetted high-error poses by 87%.

Full Paper: 3d Human Pose Estimation from 2d Keypoints (pdf)

Before After

Dependencies

Training and Testing

Training

  • Run prep_data.py
  • Run train.py, commenting out any model settings you don't want to train. A datetime is appended to these files. To use these in the subsequent step, archive the existing files and remove the datetime suffix of your new files.

Testing

Test results can be seen in model_analysis.ipynb. This notebook is not formatted with the intent of presenting the data (the research paper is), but the results can be seen there.

Prepping Data from Scratch

For the instructions in the sub-sections above, the data has already been downloaded and converted into 3d coordinates. If you want to start from scratch and convert the data from bvh files to 3d coordinates, you can proceed as follows with Blender installed:

  • The program uses CMU Motion Capture Subjects 86 for training and 13-15 for testing. Download the following groups of bvh files: Subjects 10-14, Subjects 15-19, Subjects 86-94. (main site)

  • Unzip and move folders 13, 14, 15, and 86 to directory logs/cmu_files/raw/

  • Run standardize_bvh.py to apply consistent header to each file and keep only one in every ten frames. (saved as .bvh in logs/cmu_files/standardized/)

  • Run the following to save the 3d coordinates for each frame of each corresponding animation. Blender runs the animations defined in the bvh files in the background to determine the 3d coordinates to save:

    • blender --background --python blender/save_blender_3d_points.py #Original training and testing data
    • blender --background --python blender/save_blender_mirrored.py #Bilateral mirror of Subject 86 for augmentation
    • blender logs/blender_animations/seated_animation.blend --background --python save_seated.py #Animation of seated poses for augmentation (saved as pickle dictionary in logs/blender_dicts/)

Reference

  @InProceedings{Zhao_2017,
  author = {Ruiqi Zhao, Yan Wang, and Aleix M Martines},
  title = {A Simple, Fast, and Highly-Accurate Algorithm to Recover 3D Shape from 2D Landmarks on a Single Image},
  booktitle = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  month = {November},
  year = {2017}
  }
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].