Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency

Li Zhiwei, Cai Weiling, Cairun Wang
Proceedings of The 14th Asian Conference on Machine Learning, PMLR 189:595-610, 2023.

Abstract

Unpaired photo to caricature generation is a challenging but meaningful task. Generating high quality caricatures with rich texture/color and plausible exaggeration is important. Previous methods often respectively deal with the shape transformation and texture/color style. We argue that shape transformation can be treated as same as texture/color. Thereby, shape transformation and texture/color can be transferred at the same time. In this paper, we proposed a new method namely AdsSe-GAN for photo-to-caricature generation, which consists of a new normalization function called AdaSLIN and a new semi-cycle consistency loss. The AdaSLIN adaptively selects Layer Normalization or Instance Normalization to simultaneously transfer texture/color and shape transformation. Besides we present semi-cycle consistency loss which only imposes L1 norm on caricature-to-photo process, which is different from existing methods that apply cycle consistency loss to preserve the original domain information. In fact, while generating caricature, taking no account of the cycle restriction makes our model generate caricature with more distinct exaggeration and higher quality. Experimental results on a public caricature dataset, WebCaricature, show the effectiveness of our proposed method compared with the state-of-the-art models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v189-zhiwei23a, title = {Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency}, author = {Zhiwei, Li and Weiling, Cai and Wang, Cairun}, booktitle = {Proceedings of The 14th Asian Conference on Machine Learning}, pages = {595--610}, year = {2023}, editor = {Khan, Emtiyaz and Gonen, Mehmet}, volume = {189}, series = {Proceedings of Machine Learning Research}, month = {12--14 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v189/zhiwei23a/zhiwei23a.pdf}, url = {https://proceedings.mlr.press/v189/zhiwei23a.html}, abstract = {Unpaired photo to caricature generation is a challenging but meaningful task. Generating high quality caricatures with rich texture/color and plausible exaggeration is important. Previous methods often respectively deal with the shape transformation and texture/color style. We argue that shape transformation can be treated as same as texture/color. Thereby, shape transformation and texture/color can be transferred at the same time. In this paper, we proposed a new method namely AdsSe-GAN for photo-to-caricature generation, which consists of a new normalization function called AdaSLIN and a new semi-cycle consistency loss. The AdaSLIN adaptively selects Layer Normalization or Instance Normalization to simultaneously transfer texture/color and shape transformation. Besides we present semi-cycle consistency loss which only imposes L1 norm on caricature-to-photo process, which is different from existing methods that apply cycle consistency loss to preserve the original domain information. In fact, while generating caricature, taking no account of the cycle restriction makes our model generate caricature with more distinct exaggeration and higher quality. Experimental results on a public caricature dataset, WebCaricature, show the effectiveness of our proposed method compared with the state-of-the-art models.} }
Endnote
%0 Conference Paper %T Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency %A Li Zhiwei %A Cai Weiling %A Cairun Wang %B Proceedings of The 14th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Emtiyaz Khan %E Mehmet Gonen %F pmlr-v189-zhiwei23a %I PMLR %P 595--610 %U https://proceedings.mlr.press/v189/zhiwei23a.html %V 189 %X Unpaired photo to caricature generation is a challenging but meaningful task. Generating high quality caricatures with rich texture/color and plausible exaggeration is important. Previous methods often respectively deal with the shape transformation and texture/color style. We argue that shape transformation can be treated as same as texture/color. Thereby, shape transformation and texture/color can be transferred at the same time. In this paper, we proposed a new method namely AdsSe-GAN for photo-to-caricature generation, which consists of a new normalization function called AdaSLIN and a new semi-cycle consistency loss. The AdaSLIN adaptively selects Layer Normalization or Instance Normalization to simultaneously transfer texture/color and shape transformation. Besides we present semi-cycle consistency loss which only imposes L1 norm on caricature-to-photo process, which is different from existing methods that apply cycle consistency loss to preserve the original domain information. In fact, while generating caricature, taking no account of the cycle restriction makes our model generate caricature with more distinct exaggeration and higher quality. Experimental results on a public caricature dataset, WebCaricature, show the effectiveness of our proposed method compared with the state-of-the-art models.
APA
Zhiwei, L., Weiling, C. & Wang, C.. (2023). Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency. Proceedings of The 14th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 189:595-610 Available from https://proceedings.mlr.press/v189/zhiwei23a.html.

Related Material