Enhanced security measures by 89%.

• ChallengeEfficient face anti-spoofing model
• SolutionA deep learning-based model to respond to image/video fraud attempts
• Technologies and toolsPython, deep learning, face recognition, computer vision, keypoint detection, MTCNN, TensorFlow

Client

The Client runs the business in the field of e-commerce and IT services and delivers cloud-based services to conduct financial transactions. Also, the provider offers notification and payout system for lottery players, e-payment platform, e-subscription platform, e-payment platform, and custom-tailored solutions.

The Client was looking for experts in face recognition. With the development and widespread use of computer vision-based systems, spoofing attacks have become an issue that needs solving. The Client’s business objective was to design and create a robust solution to prevent that type of attack.

Challenge: efficient face anti-spoofing model

The InData Labs’ team faced the challenge of training a deep learning model for face anti-spoofing. The Client needed the model to discern the following types of attacks:

• Printed attack (a piece of paper with a printed photo)
• Digital attack (a device with a photo)
• Replay attack (a pre-recorded video)

Our team had to design the processes of data gathering specifically for that task and collect all the data needed for training. The data for analysis and model training contained short videos taken with mobile phones’ front cameras.

Our engineers decided on using a pertained face detection network. First, we found faces and facial keypoints. Then, we used the output for spoof detection.

Solution: a deep learning-based model to respond to image/video fraud attempts

In cooperation with the Client, we prepared a dataset that contained 2500 videos representing different types of attacks along with genuine faces.

Our team started with model prototyping and then moved on to evaluating commercial implementation. We split the process of training the model into two phases: detecting static image attacks with digital/printed images and detecting video attacks.

1. Static image attacks

The task was to detect a real or fake person by using the sequence of frames or one frame from the existing dataset. For that purpose, the algorithm found key facial points (eyes, nose, corners of lips) on a range of photos. The coordinates of moving of keypoints were available to use as features.

Our engineers trained the model to classify sequences of facial keypoints as spoof/genuine to be later used for inference.

2. Digital image attacks

The task was to detect a real person or fraud attempt by using the sequence of video frames or a single frame. We decided to use different characteristics of an image: blur, color variation, and more. We used such descriptors as DOG (analysis of image distortion), LBP (texture classification), HOG (as additional features for classification). Based on those descriptors, we retrieved a set of features for training. Also, using deep learning for face anti-spoofing, we trained a separate model to scan each of the frames and then combine different types of features with the output of frame scanning within the final network.

As the final solution, we proposed the following:

• A frame-by-frame model to analyze each frame separately and extract certain features
• Data on facial keypoints to be used together with the features
• An RNN to analyze the features and dynamics of keypoints
• The model solving the task on a sequence of frames for both static and video attacks

Result: PoC and consulting on the further development of the project

The InData Labs’ team created Python-based PoC and provided the Client with a demo to tackle the face anti-spoofing challenge. We delivered the whole pipeline of model training and inference, including the option to import the final neural network so that the Client could use it from Java.

The code we delivered could be used to train models on extended data and high-speed models. We also presented several types of models.

By using our model, the Client can choose the decision threshold and find a balance between FAR (false accept rate) and TAR (true accept rate). We have achieved the quality initially requested by the Client.

The engineering team offered its solid experience in creating face recognition and other systems with computer vision as the basis. Our solution addresses the acute business need for security measures. Our Client obtained a working model answering the goals and received related recommendations from our experts on further advanced model training.