Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability

Yunshu Dai, Jianwei Fei, Fangjun Huang, Chip Hong Chang
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:12047-12062, 2025.

Abstract

As AI generative models evolve, face swap technology has become increasingly accessible, raising concerns over potential misuse. Celebrities may be manipulated without consent, and ordinary individuals may fall victim to identity fraud. To address these threats, we propose Secure Swap, a method that protects persons of interest (POI) from face-swapping abuse and embeds a unique, invisible watermark into nonPOI swapped images for traceability. By introducing an ID Passport layer, Secure Swap redacts POI faces and generates watermarked outputs for nonPOI. A detachable watermark encoder and decoder are trained with the model to ensure provenance tracing. Experimental results demonstrate that Secure Swap not only preserves face swap functionality but also effectively prevents unauthorized swaps of POI and detects different embedded model’s watermarks with high accuracy. Specifically, our method achieves a 100% success rate in protecting POI and over 99% watermark extraction accuracy for nonPOI. Besides fidelity and effectiveness, the robustness of protected models against image-level and model-level attacks in both online and offline application scenarios is also experimentally demonstrated.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-dai25f, title = {Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability}, author = {Dai, Yunshu and Fei, Jianwei and Huang, Fangjun and Chang, Chip Hong}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {12047--12062}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/dai25f/dai25f.pdf}, url = {https://proceedings.mlr.press/v267/dai25f.html}, abstract = {As AI generative models evolve, face swap technology has become increasingly accessible, raising concerns over potential misuse. Celebrities may be manipulated without consent, and ordinary individuals may fall victim to identity fraud. To address these threats, we propose Secure Swap, a method that protects persons of interest (POI) from face-swapping abuse and embeds a unique, invisible watermark into nonPOI swapped images for traceability. By introducing an ID Passport layer, Secure Swap redacts POI faces and generates watermarked outputs for nonPOI. A detachable watermark encoder and decoder are trained with the model to ensure provenance tracing. Experimental results demonstrate that Secure Swap not only preserves face swap functionality but also effectively prevents unauthorized swaps of POI and detects different embedded model’s watermarks with high accuracy. Specifically, our method achieves a 100% success rate in protecting POI and over 99% watermark extraction accuracy for nonPOI. Besides fidelity and effectiveness, the robustness of protected models against image-level and model-level attacks in both online and offline application scenarios is also experimentally demonstrated.} }
Endnote
%0 Conference Paper %T Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability %A Yunshu Dai %A Jianwei Fei %A Fangjun Huang %A Chip Hong Chang %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-dai25f %I PMLR %P 12047--12062 %U https://proceedings.mlr.press/v267/dai25f.html %V 267 %X As AI generative models evolve, face swap technology has become increasingly accessible, raising concerns over potential misuse. Celebrities may be manipulated without consent, and ordinary individuals may fall victim to identity fraud. To address these threats, we propose Secure Swap, a method that protects persons of interest (POI) from face-swapping abuse and embeds a unique, invisible watermark into nonPOI swapped images for traceability. By introducing an ID Passport layer, Secure Swap redacts POI faces and generates watermarked outputs for nonPOI. A detachable watermark encoder and decoder are trained with the model to ensure provenance tracing. Experimental results demonstrate that Secure Swap not only preserves face swap functionality but also effectively prevents unauthorized swaps of POI and detects different embedded model’s watermarks with high accuracy. Specifically, our method achieves a 100% success rate in protecting POI and over 99% watermark extraction accuracy for nonPOI. Besides fidelity and effectiveness, the robustness of protected models against image-level and model-level attacks in both online and offline application scenarios is also experimentally demonstrated.
APA
Dai, Y., Fei, J., Huang, F. & Chang, C.H.. (2025). Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:12047-12062 Available from https://proceedings.mlr.press/v267/dai25f.html.

Related Material