Weighted quantization using MMD: From mean field to mean shift using gradient flows

Ayoub Belhadji, Daniel Sharp, Youssef Marzouk
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:1045-1053, 2026.

Abstract

Approximating a probability distribution using a set of particles is a fundamental problem in machine learning and statistics, with applications including clustering and quantization. Formally, we seek a weighted mixture of Dirac measures that best approximates the target distribution. While much existing work relies on the Wasserstein distance to quantify approximation errors, maximum mean discrepancy (MMD) has received comparatively less attention, especially when allowing for variable particle weights. We argue that a \emph{Wasserstein–Fisher–Rao} gradient flow is well-suited for designing quantizations optimal under MMD. We show that a system of interacting particles satisfying a set of ODEs discretizes this flow. We further derive a new fixed-point algorithm called \emph{mean shift interacting particles} (MSIP). We show that MSIP extends the classical mean shift algorithm, widely used for identifying modes in kernel density estimators. Moreover, we show that MSIP can be interpreted as preconditioned gradient descent and that it acts as a relaxation of Lloyd’s algorithm for clustering. Our unification of gradient flows, mean shift, and MMD-optimal quantization yields algorithms that are more robust than state-of-the-art methods, as demonstrated via high-dimensional and multi-modal numerical experiments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-belhadji26a, title = { Weighted quantization using MMD: From mean field to mean shift using gradient flows }, author = {Belhadji, Ayoub and Sharp, Daniel and Marzouk, Youssef}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {1045--1053}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/belhadji26a/belhadji26a.pdf}, url = {https://proceedings.mlr.press/v300/belhadji26a.html}, abstract = { Approximating a probability distribution using a set of particles is a fundamental problem in machine learning and statistics, with applications including clustering and quantization. Formally, we seek a weighted mixture of Dirac measures that best approximates the target distribution. While much existing work relies on the Wasserstein distance to quantify approximation errors, maximum mean discrepancy (MMD) has received comparatively less attention, especially when allowing for variable particle weights. We argue that a \emph{Wasserstein–Fisher–Rao} gradient flow is well-suited for designing quantizations optimal under MMD. We show that a system of interacting particles satisfying a set of ODEs discretizes this flow. We further derive a new fixed-point algorithm called \emph{mean shift interacting particles} (MSIP). We show that MSIP extends the classical mean shift algorithm, widely used for identifying modes in kernel density estimators. Moreover, we show that MSIP can be interpreted as preconditioned gradient descent and that it acts as a relaxation of Lloyd’s algorithm for clustering. Our unification of gradient flows, mean shift, and MMD-optimal quantization yields algorithms that are more robust than state-of-the-art methods, as demonstrated via high-dimensional and multi-modal numerical experiments. } }
Endnote
%0 Conference Paper %T Weighted quantization using MMD: From mean field to mean shift using gradient flows %A Ayoub Belhadji %A Daniel Sharp %A Youssef Marzouk %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-belhadji26a %I PMLR %P 1045--1053 %U https://proceedings.mlr.press/v300/belhadji26a.html %V 300 %X Approximating a probability distribution using a set of particles is a fundamental problem in machine learning and statistics, with applications including clustering and quantization. Formally, we seek a weighted mixture of Dirac measures that best approximates the target distribution. While much existing work relies on the Wasserstein distance to quantify approximation errors, maximum mean discrepancy (MMD) has received comparatively less attention, especially when allowing for variable particle weights. We argue that a \emph{Wasserstein–Fisher–Rao} gradient flow is well-suited for designing quantizations optimal under MMD. We show that a system of interacting particles satisfying a set of ODEs discretizes this flow. We further derive a new fixed-point algorithm called \emph{mean shift interacting particles} (MSIP). We show that MSIP extends the classical mean shift algorithm, widely used for identifying modes in kernel density estimators. Moreover, we show that MSIP can be interpreted as preconditioned gradient descent and that it acts as a relaxation of Lloyd’s algorithm for clustering. Our unification of gradient flows, mean shift, and MMD-optimal quantization yields algorithms that are more robust than state-of-the-art methods, as demonstrated via high-dimensional and multi-modal numerical experiments.
APA
Belhadji, A., Sharp, D. & Marzouk, Y.. (2026). Weighted quantization using MMD: From mean field to mean shift using gradient flows . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:1045-1053 Available from https://proceedings.mlr.press/v300/belhadji26a.html.

Related Material