Skip to content

SFI-Swin: Symmetric Face Inpainting with Swin Transformer by Distinctly Learning Face Components Distributions https://arxiv.org/abs/2301.03130

License

Notifications You must be signed in to change notification settings

mohammadrezanaderi4/SFI-Swin

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SFI-Swin: Symmetric Face Inpainting with Swin Transformer by Distinctly Learning Face Components Distributions

arXiv GitHub Stars visitors
[Project page]

Official Pytorch implementation

by MohammadReza Naderi , MohammadHossein Givkashi , Nader Karimi , Shahram Shirani , Shadrokh Samavi



Implementations

  • Coming Soon ...

Blockdiagram

What is the general explainations of the proposed method?
First, the generator takes the masked image as input and attempts to inpaint it. Then the inpainted image is fed to the patch discriminator to check the overall reality of the patches. Meanwhile, the inpainted image is also fed into a semantic segmentation network to separate the semantic parts of the face, such as eyes, and ears. (The architecture of these six semantic discriminators is the same). In the next step, six distinct discriminators calculate the total realness of each semantic part of the face. Then, the generator parameters will be updated based on these seven discriminators and the pixel-wise loss gradient signals, which are backpropagated to the generator.

FID and LPIPS comparison

SCS comparison

what is symmetry concentration score (SCS)?
Symmetry concentration Score is a newly introduce metric which explained compeletly in the paper to measure the symmerty of the repaired faces base on the concentration of the inpaintor.
How to calculate it?
First, we mask an eye and a K×K patch of the face, and reconstruct the missing eye and the K×K patch. Then the absolute difference between the image with inpainted eye with the image with a missing K×K patch and the missing eye is computed. The difference shows the effect of that K×K patch on inpainting result of the missing eye. The effect of all K×K patches is computed and shown as a heatmap. The face borders are also depicted to investigate the impact of each part of the face on inpainting the missed eye.

Acknowledgments

Citation

If you found this code helpful, please consider citing:

@article{naderi2023sfi,
  title={SFI-Swin: Symmetric Face Inpainting with Swin Transformer by Distinctly Learning Face Components Distributions},
  author={Naderi, MohammadReza and Givkashi, MohammadHossein and Karimi, Nader and Shirani, Shahram and Samavi, Shadrokh},
  journal={arXiv preprint arXiv:2301.03130},
  year={2023}
}

Releases

No releases published

Packages

No packages published