Multiobjective distribution matching

Xiaoyuan Zhang, Peijie Li, Yingying Yu, Yichi Zhang, Han Zhao, Qingfu Zhang
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:75413-75426, 2025.

Abstract

Distribution matching is a key technique in machine learning, with applications in generative models, domain adaptation, and algorithmic fairness. A related but less explored challenge is generating a distribution that aligns with multiple underlying distributions, often with conflicting objectives, known as a Pareto optimal distribution. In this paper, we develop a general theory based on information geometry to construct the Pareto set and front for the entire exponential family under KL and inverse KL divergences. This formulation allows explicit derivation of the Pareto set and front for multivariate normal distributions, enabling applications like multiobjective variational autoencoders (MOVAEs) to generate interpolated image distributions. Experimental results on real-world images demonstrate that both algorithms can generate high-quality interpolated images across multiple distributions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-zhang25as, title = {Multiobjective distribution matching}, author = {Zhang, Xiaoyuan and Li, Peijie and Yu, Yingying and Zhang, Yichi and Zhao, Han and Zhang, Qingfu}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {75413--75426}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/zhang25as/zhang25as.pdf}, url = {https://proceedings.mlr.press/v267/zhang25as.html}, abstract = {Distribution matching is a key technique in machine learning, with applications in generative models, domain adaptation, and algorithmic fairness. A related but less explored challenge is generating a distribution that aligns with multiple underlying distributions, often with conflicting objectives, known as a Pareto optimal distribution. In this paper, we develop a general theory based on information geometry to construct the Pareto set and front for the entire exponential family under KL and inverse KL divergences. This formulation allows explicit derivation of the Pareto set and front for multivariate normal distributions, enabling applications like multiobjective variational autoencoders (MOVAEs) to generate interpolated image distributions. Experimental results on real-world images demonstrate that both algorithms can generate high-quality interpolated images across multiple distributions.} }
Endnote
%0 Conference Paper %T Multiobjective distribution matching %A Xiaoyuan Zhang %A Peijie Li %A Yingying Yu %A Yichi Zhang %A Han Zhao %A Qingfu Zhang %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-zhang25as %I PMLR %P 75413--75426 %U https://proceedings.mlr.press/v267/zhang25as.html %V 267 %X Distribution matching is a key technique in machine learning, with applications in generative models, domain adaptation, and algorithmic fairness. A related but less explored challenge is generating a distribution that aligns with multiple underlying distributions, often with conflicting objectives, known as a Pareto optimal distribution. In this paper, we develop a general theory based on information geometry to construct the Pareto set and front for the entire exponential family under KL and inverse KL divergences. This formulation allows explicit derivation of the Pareto set and front for multivariate normal distributions, enabling applications like multiobjective variational autoencoders (MOVAEs) to generate interpolated image distributions. Experimental results on real-world images demonstrate that both algorithms can generate high-quality interpolated images across multiple distributions.
APA
Zhang, X., Li, P., Yu, Y., Zhang, Y., Zhao, H. & Zhang, Q.. (2025). Multiobjective distribution matching. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:75413-75426 Available from https://proceedings.mlr.press/v267/zhang25as.html.

Related Material