MagicMask: A Fast and High-fidelity Face Swapping Method Robust to Face Pose

Jongmin Yu, Anoushka Harit, Jiankang Deng, Shan Luo, Jinhong Yang, Zhongtian Sun
Proceedings of the 17th Asian Conference on Machine Learning, PMLR 304:1054-1069, 2025.

Abstract

Recent face-swapping methods excel under controlled conditions but often fail when presented with extreme facial poses. Diffusion-based approaches may be able to overcome these issues, but they still face significant computational costs. This paper introduces MagicMask, a novel face-swapping framework that robustly handles various poses in real time by fusing visual and geometric information. Our method incorporates explicit, identity-adapted geometric cues into the latent feature space via a multi-head attention mechanism. It employs an Adversarial Facial Silhouette Alignment (AFSA) loss to preserve detailed facial boundaries adapted to source identity. Comprehensive experiments on multiple benchmarks demonstrate that MagicMask competes with state-of-the-art methods under standard conditions and significantly outperforms them in extreme pose scenarios. The source code for the demonstration of MagicMask is attached as supplementary materials.

Cite this Paper

BibTeX
@InProceedings{pmlr-v304-yu25a, title = {MagicMask: A Fast and High-fidelity Face Swapping Method Robust to Face Pose}, author = {Yu, Jongmin and Harit, Anoushka and Deng, Jiankang and Luo, Shan and Yang, Jinhong and Sun, Zhongtian}, booktitle = {Proceedings of the 17th Asian Conference on Machine Learning}, pages = {1054--1069}, year = {2025}, editor = {Lee, Hung-yi and Liu, Tongliang}, volume = {304}, series = {Proceedings of Machine Learning Research}, month = {09--12 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v304/main/assets/yu25a/yu25a.pdf}, url = {https://proceedings.mlr.press/v304/yu25a.html}, abstract = {Recent face-swapping methods excel under controlled conditions but often fail when presented with extreme facial poses. Diffusion-based approaches may be able to overcome these issues, but they still face significant computational costs. This paper introduces MagicMask, a novel face-swapping framework that robustly handles various poses in real time by fusing visual and geometric information. Our method incorporates explicit, identity-adapted geometric cues into the latent feature space via a multi-head attention mechanism. It employs an Adversarial Facial Silhouette Alignment (AFSA) loss to preserve detailed facial boundaries adapted to source identity. Comprehensive experiments on multiple benchmarks demonstrate that MagicMask competes with state-of-the-art methods under standard conditions and significantly outperforms them in extreme pose scenarios. The source code for the demonstration of MagicMask is attached as supplementary materials.} }
Endnote
%0 Conference Paper %T MagicMask: A Fast and High-fidelity Face Swapping Method Robust to Face Pose %A Jongmin Yu %A Anoushka Harit %A Jiankang Deng %A Shan Luo %A Jinhong Yang %A Zhongtian Sun %B Proceedings of the 17th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Hung-yi Lee %E Tongliang Liu %F pmlr-v304-yu25a %I PMLR %P 1054--1069 %U https://proceedings.mlr.press/v304/yu25a.html %V 304 %X Recent face-swapping methods excel under controlled conditions but often fail when presented with extreme facial poses. Diffusion-based approaches may be able to overcome these issues, but they still face significant computational costs. This paper introduces MagicMask, a novel face-swapping framework that robustly handles various poses in real time by fusing visual and geometric information. Our method incorporates explicit, identity-adapted geometric cues into the latent feature space via a multi-head attention mechanism. It employs an Adversarial Facial Silhouette Alignment (AFSA) loss to preserve detailed facial boundaries adapted to source identity. Comprehensive experiments on multiple benchmarks demonstrate that MagicMask competes with state-of-the-art methods under standard conditions and significantly outperforms them in extreme pose scenarios. The source code for the demonstration of MagicMask is attached as supplementary materials.
APA
Yu, J., Harit, A., Deng, J., Luo, S., Yang, J. & Sun, Z.. (2025). MagicMask: A Fast and High-fidelity Face Swapping Method Robust to Face Pose. Proceedings of the 17th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 304:1054-1069 Available from https://proceedings.mlr.press/v304/yu25a.html.

Related Material