Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes

Dongjae Jeon, Dueun Kim, Albert No
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:27091-27112, 2025.

Abstract

In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-jeon25a, title = {Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes}, author = {Jeon, Dongjae and Kim, Dueun and No, Albert}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {27091--27112}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/jeon25a/jeon25a.pdf}, url = {https://proceedings.mlr.press/v267/jeon25a.html}, abstract = {In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion.} }
Endnote
%0 Conference Paper %T Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes %A Dongjae Jeon %A Dueun Kim %A Albert No %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-jeon25a %I PMLR %P 27091--27112 %U https://proceedings.mlr.press/v267/jeon25a.html %V 267 %X In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion.
APA
Jeon, D., Kim, D. & No, A.. (2025). Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:27091-27112 Available from https://proceedings.mlr.press/v267/jeon25a.html.

Related Material