HumanRecognition
The system is develped to perform person recognition task on PIPA dataset, the detailed description of the approach of this system can be found here.
Performance (Last Updated: 5/19/2017)
| Config | Accuracy |
|---|---|
| Face | 62.18% |
| Head | 63.19% |
| Upper-body | 67.44% |
| Full-body | 58.96% |
| All modality fused | 82.31% |
| All modality fused + MRF refining | 86.18% |
Usage
Preparation
-
get dataset
./scripts/get_PIPA.sh -
get models
./scripts/get_models.sh -
get features (if you don't wish to extract them)
./scripts/get_features.sh
Face Feature
We use FaceNet for face feature extraction. FaceNet is a CNN trained to directly optimize the embedding itself.
-
test face feature extractor
python pyHumanRecog/face_feature_extractor_test.py
Head Feature
-
train head feature extractor (feel free to experiment with different batch size)
python pyHumanRecog/head_feature_extractor_train.py --batch_size 32 -
test head feature extractor
python pyHumanRecog/head_feature_extractor_test.py --batch_size 32
Body Feature
-
train body feature extractor (feel free to experiment with different batch size)
python pyHumanRecog/body_feature_extractor_train.py --batch_size 32 -
test body feature extractor
python pyHumanRecog/body_feature_extractor_test.py --batch_size 32
Upper-body Feature
-
train upper-body feature extractor (feel free to experiment with different batch size)
python pyHumanRecog/upper_body_feature_extractor_train.py --batch_size 32 -
test upper-body feature extractor
python pyHumanRecog/upper_body_feature_extractor_test.py --batch_size 32
Pose estimation
We use CPM for pose estimation. The estimated CPM pose will mainly be used for image warping.
-
CPM pose estimation
python pyHumanRecog/extract_pose.py <img_dump_folder> <pose_dump_folder><image_dump_folder>: folder to dump CPM pose visualization images<pose_dump_folder>: folder to dump CPM pose positions
Evaluation
For performance evaluation, Please first modify performance_test_config.py (within pyHumanRecog folder) to specify the features you wish to use and their corresponding weights. Then execute the following command.
python pyHumanRecog/performance_test.py
To perform MRF optimization (which incorporates the photo-level cooccurrence and mutual exclusive pattern into the final prediction), Set refine_with_photo_level_context = True in HumanRecog/performance_test_config.py.