[edit]
Sliced Wasserstein variational inference
Proceedings of The 14th Asian Conference on Machine
Learning, PMLR 189:1213-1228, 2023.
Abstract
Variational Inference approximates an unnormalized
distribution via the minimization of
Kullback-Leibler (KL) divergence. Although this
divergence is efficient for computation and has been
widely used in applications, it suffers from some
unreasonable properties. For example, it is not a
proper metric, i.e., it is non-symmetric and does
not preserve the triangle inequality. On the other
hand, optimal transport distances recently have
shown some advantages over KL divergence. With the
help of these advantages, we propose a new
variational inference method by minimizing sliced
Wasserstein distance–a valid metric arising from
optimal transport. This sliced Wasserstein distance
can be approximated simply by running MCMC but
without solving any optimization problem. Our
approximation also does not require a tractable
density function of variational distributions so
that approximating families can be amortized by
generators like neural networks. Furthermore, we
provide an analysis of the theoretical properties of
our method. Experiments on synthetic and real data
are illustrated to show the performance of the
proposed method.