Mixup Regularization: A Probabilistic Perspective

Yousef El-Laham, Niccolo Dalmasso, Svitlana Vyetrenko, Vamsi K. Potluru, Manuela Veloso
Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, PMLR 286:1184-1219, 2025.

Abstract

In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants.

Cite this Paper

BibTeX
@InProceedings{pmlr-v286-el-laham25a, title = {Mixup Regularization: A Probabilistic Perspective}, author = {El-Laham, Yousef and Dalmasso, Niccolo and Vyetrenko, Svitlana and Potluru, Vamsi K. and Veloso, Manuela}, booktitle = {Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence}, pages = {1184--1219}, year = {2025}, editor = {Chiappa, Silvia and Magliacane, Sara}, volume = {286}, series = {Proceedings of Machine Learning Research}, month = {21--25 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v286/main/assets/el-laham25a/el-laham25a.pdf}, url = {https://proceedings.mlr.press/v286/el-laham25a.html}, abstract = {In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants.} }
Endnote
%0 Conference Paper %T Mixup Regularization: A Probabilistic Perspective %A Yousef El-Laham %A Niccolo Dalmasso %A Svitlana Vyetrenko %A Vamsi K. Potluru %A Manuela Veloso %B Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2025 %E Silvia Chiappa %E Sara Magliacane %F pmlr-v286-el-laham25a %I PMLR %P 1184--1219 %U https://proceedings.mlr.press/v286/el-laham25a.html %V 286 %X In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants.
APA
El-Laham, Y., Dalmasso, N., Vyetrenko, S., Potluru, V.K. & Veloso, M.. (2025). Mixup Regularization: A Probabilistic Perspective. Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 286:1184-1219 Available from https://proceedings.mlr.press/v286/el-laham25a.html.

Related Material