Skip to content

Fast and accurate face landmark detection library using PyTorch; Support 68-point semi-frontal and 39-point profile landmark detection; Support both coordinate-based and heatmap-based inference; Up to 100 FPS landmark inference speed with SOTA face detector on CPU.

Notifications You must be signed in to change notification settings

cunjian/pytorch_face_landmark

Repository files navigation

Pytorch Face Landmark Detection

Implementation of face landmark detection with PyTorch. The models were trained using coordinate-based or heatmap-based regression methods. Different face detetors were supported. A video demo and image detection results were displayed here.

  • Support 68-point and 39-point landmark inference.
  • Support automatic alignment and crop
  • Support different backbone networks and face detectors.
  • Support ONNX and OpenVINO inferences.
  • Support heatmap-based models.

Inference

Test on a sample folder and save the landmark detection and cropped face results. Support different backbones and face detectors.

python3 test_batch_detections.py --backbone MobileFaceNet --detector Retinaface

  • backbone: MobileNet/PFLD/MobileFaceNet; detector: MTCNN/FaceBoxes/Retinaface

Optimize with ONNX and test on a camera with MTCNN as a face detector.

python3 test_camera_mtcnn_onnx.py

Optimize with ONNX and test on a camera with a lightweight face detector. It can achieve real-time speed on CPU.

python3 test_camera_light_onnx.py

Optimize with OpenVINO and test on a camera with a lightweight face detector. About 10 times faster than ONNX.

python3 test_camera_mobilefacenet_openvino.py

Benchmark Results on 300W

  • Inter-ocular Normalization (ION)
Algorithms Common Challenge Full Set Param # (M)
ResNet18 (224×224) 3.73 7.14 4.39 11.76
Res2Net50_v1b (224×224) 3.43 6.77 4.07 26.00
Res2Net50_v1b_SE (224×224) 3.37 6.67 4.01 27.05
Res2Net50_v1b_ExternalData (224×224) 3.30 5.92 3.81 26.00
HRNet_w18_small_v2 (224×224) 3.57 6.85 4.20 13.83
  • Inter-ocular Normalization (ION) with Lightweight Models
Algorithms Common Challenge Full Set Param # (M) CPU Inference (s)
MobileNetV2 (224×224) 3.70 7.27 4.39 3.74 1.2
MobileNetV2_SE (224×224) 3.63 7.01 4.28 4.15 /
MobileNetV2_SE_RE (224×224) 3.63 6.66 4.21 4.15 /
MobileNetV2_ExternalData (224×224) 3.48 6.0 3.96 3.74 1.2
MobileNetV2 (56×56) 4.50 8.50 5.27 3.74 0.01 (onnx)
MobileNetV2_SE_ExternalData (56×56) 4.10 6.89 4.64 4.10 0.01 (onnx)
PFLD_ExternalData (112×112) 3.49 6.01 3.97 0.73 0.01 (onnx)
MobileFaceNet_ExternalData (112×112) 3.30 5.69 3.76 1.01 /

Note: SE (squeeze-and-excitation module), RE (random erasing module).

  • Heatmap Inference (still under test)
Algorithms Common Challenge Full Set Param # (M)
Hourglass2 3.06 5.54 3.55 8.73

Visualization Results

  • Face alignment on 300W dataset

img1

  • Semi-frontal face alignment on Menpo dataset

img1

  • Profile face alignment on Menpo dataset

img1

TODO

The following features will be added soon.

  • Still to come:
    • Support for the 39-point detection
    • Support for the 106 point detection
    • Support for heatmap-based inferences

Public Datasets:

Citations:

If you use library in your project, please, cite it as follows.

@misc{PFL,
  title={{PyTorch Face Landmark}: A Fast and Accurate Facial Landmark Detector},
  url={https://github.com/cunjian/pytorch_face_landmark},
  note={Open-source software available at https://github.com/cunjian/pytorch_face_landmark},
  author={Cunjian Chen},
  year={2021},
}

References:

About

Fast and accurate face landmark detection library using PyTorch; Support 68-point semi-frontal and 39-point profile landmark detection; Support both coordinate-based and heatmap-based inference; Up to 100 FPS landmark inference speed with SOTA face detector on CPU.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages