Dataset Dynamics via Gradient Flows in Probability Space

David Alvarez-Melis, Nicolò Fusi
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:219-230, 2021.

Abstract

Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-alvarez-melis21a, title = {Dataset Dynamics via Gradient Flows in Probability Space}, author = {Alvarez-Melis, David and Fusi, Nicol\`o}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {219--230}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/alvarez-melis21a/alvarez-melis21a.pdf}, url = {https://proceedings.mlr.press/v139/alvarez-melis21a.html}, abstract = {Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.} }
Endnote
%0 Conference Paper %T Dataset Dynamics via Gradient Flows in Probability Space %A David Alvarez-Melis %A Nicolò Fusi %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-alvarez-melis21a %I PMLR %P 219--230 %U https://proceedings.mlr.press/v139/alvarez-melis21a.html %V 139 %X Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify {—}with high accuracy{—} previously unseen datasets.
APA
Alvarez-Melis, D. & Fusi, N.. (2021). Dataset Dynamics via Gradient Flows in Probability Space. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:219-230 Available from https://proceedings.mlr.press/v139/alvarez-melis21a.html.

Related Material