Awesome Human Motion

🏃‍♀️ A curated list about human motion capture, analysis and synthesis.

Contents

• Introduction
• Human Models
• Datasets
• Data Processing
• Pose Estimation
• Motion Analysis
• Motion Synthesis
• Researchers, Institutes, Projects
• Commercial Projects
• Journals
• Conferences
• Videos

Introduction

• Human Pose and Motion - A gentle introduction.

Human Models

• SMPL - SMPL is a realistic 3D model of the human body that is based on skinning and blend shapes and is learned from thousands of 3D body scans.
• MakeHuman - MakeHuman is an open source (AGPL3) tool designed to simplify the creation of virtual humans using a Graphical User Interface, also commonly referred to as a GUI.

Datasets

• Human 3.6M - Large Scale Datasets and Predictive Methodsfor 3D Human Sensing in Natural Environments
• SURREAL - Learning from Synthetic Humans, CVPR 2017
• CMU - Carnegie Mellon University Motion Capture Database
• Berkley MHAD - [📷🎥🎤🤾‍♀️⌚️🤹‍♀️][👨‍🦰👩][👧👵] - The Berkeley Multimodal Human Action Database (MHAD) contains 11 actions performed by 7 male and 5 female subjects in the range 23-30 years of age except for one elderly subject.
• COCO - [📷][👨‍🦰👩][👧👵] - COCO is a large-scale object detection, segmentation, and captioning dataset.
• HDM05 - HDM05 contains more than three hours of systematically recorded and well-documented motion capture data in the C3D as well as in the ASF/AMC data format.
• KIT Whole-Body Human Motion Database
• CGVU Interaction Database - This is the project page for creating a database of interactions between a character and an object/objects.

Description

• Sensors and Data Types - 📷(image), 🎥(video), 🎤(audio), 🤾‍♀️(Motion Capture), ⌚️(IMU or wearables), 🤹‍♀️(Kinect or similar)
• Sex - 👨‍🦰(male), 👩(female)
• Age - 👧(young), 👵(eldery)

Data Processing

Recording

• Unity Humanoid Mocap CSV - Use .csv files to record, play and evaluate motion capture data.
• KinectMotionCapture - A simple software for capturing human body movements using the Kinect camera.

Data Conversion

• video to bvh - Convert human motion from video to .bvh.
• MotionCapturePy - Converts motion capture data from ASF and AMC files to Cartesian numpy arrays in python. Also plots a moving human frame using matplotlib.

Misc

• Motion Annotation Tool - Crowd-sourced Annotation of Human Motion.

Pose Estimation

Lectures

• Human Pose Estimation 101 - Basics of 2D and 3D Human Pose Estimation.
• Object Keypoint Similarity - This page describes the keypoint evaluation metrics used by COCO.

Papers

• Papers with Code - A collection of papers addressing several tasks of pose estimation with code available.
• Human Pose Estimation Papers - A collection of papers addressing 2D and 3D human pose estimation.

Implementations

• 3Dpose_ssl - 3D Human Pose Machines with Self-supervised Learning.
• 3dpose_gan - The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations.
• 3d_pose_baseline_pytorch - A simple baseline for 3d human pose estimation in PyTorch.
• 3d-pose-estimation - VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera.
• 3D-HourGlass-Network - 3D HourGlass Networks for Human Pose Estimation Through Videos.
• adversarially_parameterized_optimization - GAN-based 3D human pose estimation.
• DensePose - A real-time approach for mapping all human pixels of 2D RGB images to a 3D surface-based model of the body
• VideoPose3D - Efficient 3D human pose estimation in video using 2D keypoint trajectorie.
• 3d-pose-baseline - A simple baseline for 3d human pose estimation in tensorflow. Presented at ICCV 17.
• Human Shape and Pose - End-to-end Recovery of Human Shape and Pose - CVPR 2018
• AlphaPose - Real-Time and Accurate Multi-Person Pose Estimation&Tracking System.

Motion Analysis

Implementations

• GaitAnalysisToolKit - Tools for the Cleveland State Human Motion and Control Lab.
• motion classification - The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".
• Human-detection-system-with-raspberry-Pi - A motion detection system with RaspberryPi, OpenCV, Python.
• humanMotionClassification - Experiments in classifying human actions using the UCF action databased.
• sensormotion - Python package for analyzing sensor-collected human motion data (e.g. physical activity levels, gait dynamics).
• Posture and Fall Detection System Using 3D Motion Sensors - This work presents a supervised learning approach for training a posture detection classifier, and implementing a fall detection system using the posture classification results as inputs with a Microsoft Kinect v2 sensor.
• HumanMotionVisualiser - This project is for visualising human motion data captured from Kinect V2 for further data analysis.
• human motion analysis
• human motion classification
• motion visualization - A simple visualizer for human whole-body motion using three.js

Motion Synthesis

Implementations

• Auto Conditioned RNN motion - Implementation of Auto-Conditioned Recurrent Networks for Extended Complex Human Motion Synthesis.
• Character Animation - A Re-implementation of the paper "A Deep Learning Framework for Character Motion Synthesis and Editing".
• eccv18_mtvae - Tensorflow Implementation of ECCV'18 paper: Multimodal Human Motion Synthesis.
• motionSynth - Deep Human Motion Synthesis.
• MotionSynthesis2Maya - create a Maya Pipeline based on Motion Synthesis.
• Adversarial Learning for Modeling Human Motion - This repository contains the open source code which reproduces the results for the paper: Adversarial learning for modeling human motion.
• Human Motion Synthesis - Human motion synthesis using Unity3D.
• GAN motion Prediction - An LSTM based GAN for Human motion synthesis.
• Merel MoCap GAIL - An implementation of "Learning human behaviors from motion capture by adversarial imitation".
• Skills from Videos - Motion Reconstruction Code and Data for Skills from Videos (SFV)

Researchers, Institutes, Projects

People

• Daniel Holden - My name is Daniel Holden. I'm a programmer, writer, and digital artist currently working as a Machine Learning researcher at Ubisoft Montreal. My interests are Computer Graphics, Game Development, Theory of Computation, and Programming Languages.
• Gustavo Boehs (3DeepLearner) - Deep Learning for Technical Artists in Animation, VFX, and Games.
• Arash Hosseini - R&D Engineer and ML Enthusiast.
• Sebastian Starke - Ph.D. student in Character Animation and Artificial Intelligence at the University of Edinburgh, School of Informatics, Institute of Perception, Action and Behaviour, supervised by Dr. Taku Komura.

Institutes and Projects

• Uni Bonn: Physics-based motion analysis and synthesis - Physically-based analysis and synthesis of (human) motions have a number of applications. They can help to enhance the efficiency of medical rehabilitation, to improve the understanding of motions in the realm of sports or to generate realistic animations for movies.
• Max Planck Institute for Intelligent Systems: Perceiving Systems - We combine research on computer vision, computer graphics, and machine learning to teach computers to see us and by understanding our behavior learn to be more human.
• FAU: Biomechanical Simulation for the Reconstruction and Synthesis of Human Motion - In this project, we investigate musculoskeletal modeling and simulation to analyze and understand human movement and performance. Our objective is to reconstruct human motion from measurement data for example for medical assessments or to predict human responses for virtual product development.
• USC Institute for Creative Technologies: SmartBody - SmartBody is a character animation platform originally developed at the USC Institute for Creative Technologies. SmartBody provides locomotion, steering, object manipulation, lip syncing, gazing, nonverbal behavior and retargeting in real time.

Commercial Projects

• wrnch.ai - wrnch is a computer vision / deep learning software engineering company based in Montréal, Canada, a renowned hub for AI.
• Qinematic - Qinematic has developed software for 3D markerless motion capture and human movement analysis since 2012.
• DeepMotion - DeepMotion’s solutions bridge physical and digital motion for virtual characters and machines. Using physics simulation, computer vision, and machine learning, DeepMotion reconstructs realistic motion from real-world examples.

Journals

• Computers in Biology and Medicine
• Informatics in Medicine Unlocked
• Image and Vision Computing
• Clinical Biomechanics
• Signal Processing: Image Communication
• Artificial Intelligence in Medicine

Journal Ranking

• Google Scholar - Top publications by category and/or language.

Conferences

• ICRA - International Conference on Robotics and Automation
• MICCAI - Medical Image Computing and Computer-Assisted Intervention
• CVF - The Computer Vision Foundation

Conference Ranking

• Conference Rank - Look up the rank of your conference.
• Google Scholar - Top publications by category and/or language.

Videos

Two Minute Papers

• We Can All Become Video Game Characters With This AI - The paper "Vid2Game: Controllable Characters Extracted from Real-World Videos" is available here: https://arxiv.org/abs/1904.08379

CVPR 2019

• Oral Session 2-1C: Motion & Biometrics
• Oral Session 1-1C: Action & Video
• Oral Session 1-2B: Synthesis
• Oral Session 3-1A: Applications
• Oral Session 3-2B: Face & Body

Credits

This list benefits massively from the research work of Loreen Pogrzeba.

Contribute

Contributions welcome! Read the contribution guidelines first.

License

To the extent possible under law, derikon has waived all copyright and related or neighboring rights to this work.