Broutonlab face recognition with medical masks

This repository contains the source code for the article on Face recognition with medical masks by Alexey Kovalenko and Artem Poltavskiy

Pipeline with training face recognition

The whole pipeline code for training with detailed description provided in google colab notebook.

Test medical masks augmentations

You can also test masked faces pipeline from this colab notebook

Article abstract

Struggle

Identification systems which is we use for unlocking our devices have struggled with medical masks appearing on human faces.

Solution

We will show and build system with the most modern state-of-the-art methods possible to solve the task of face recognition with medical masks.  In order to do that, we will make such augmentations that transform our initial training dataset into persons wearing medical masks.

Process of facial keypoints extraction

Triangulation process

Medical mask matching

Situation with the face rotation

Proposed solution also handles the situation with the face rotation, as medical masks database is stored in json with the calculated parameter of rotation, which allow us to match images with face rotation for only with those masks that are falling in concrete interval of rotation for given face.

ArcFace

Process of training a DCNN for face recognition supervised by the ArcFace loss

Results

We were able to achieve 58 percents accuracy with custom metric on test dataset. The ability to show impressive results for such limited training time proves that pipeline is able to solve face recognition with medical masks task.