A Note on Metric Properties for Some Divergence Measures: The Gaussian Case

Karim T. Abou-Moustafa, Frank P. Ferrie
Proceedings of the Asian Conference on Machine Learning, PMLR 25:1-15, 2012.

Abstract

Multivariate Gaussian densities are pervasive in pattern recognition and machine learning. A central operation that appears in most of these areas is to measure the difference between two multivariate Gaussians. Unfortunately, traditional measures based on the Kullback-Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics. Next, we show how these metric axioms impact the unfolding process of manifold learning algorithms. Finally, we illustrate the efficacy of the proposed metrics on two different manifold learning algorithms when used for motion clustering in video data. Our results show that, in this particular application, the new proposed metrics lead to boosts in performance (at least 7%) when compared to other divergence measures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v25-aboumoustafa12, title = {A Note on Metric Properties for Some Divergence Measures: The Gaussian Case}, author = {Abou-Moustafa, Karim T. and Ferrie, Frank P.}, booktitle = {Proceedings of the Asian Conference on Machine Learning}, pages = {1--15}, year = {2012}, editor = {Hoi, Steven C. H. and Buntine, Wray}, volume = {25}, series = {Proceedings of Machine Learning Research}, address = {Singapore Management University, Singapore}, month = {04--06 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v25/aboumoustafa12/aboumoustafa12.pdf}, url = {https://proceedings.mlr.press/v25/aboumoustafa12.html}, abstract = {Multivariate Gaussian densities are pervasive in pattern recognition and machine learning. A central operation that appears in most of these areas is to measure the difference between two multivariate Gaussians. Unfortunately, traditional measures based on the Kullback-Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics. Next, we show how these metric axioms impact the unfolding process of manifold learning algorithms. Finally, we illustrate the efficacy of the proposed metrics on two different manifold learning algorithms when used for motion clustering in video data. Our results show that, in this particular application, the new proposed metrics lead to boosts in performance (at least 7%) when compared to other divergence measures.} }
Endnote
%0 Conference Paper %T A Note on Metric Properties for Some Divergence Measures: The Gaussian Case %A Karim T. Abou-Moustafa %A Frank P. Ferrie %B Proceedings of the Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2012 %E Steven C. H. Hoi %E Wray Buntine %F pmlr-v25-aboumoustafa12 %I PMLR %P 1--15 %U https://proceedings.mlr.press/v25/aboumoustafa12.html %V 25 %X Multivariate Gaussian densities are pervasive in pattern recognition and machine learning. A central operation that appears in most of these areas is to measure the difference between two multivariate Gaussians. Unfortunately, traditional measures based on the Kullback-Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics. Next, we show how these metric axioms impact the unfolding process of manifold learning algorithms. Finally, we illustrate the efficacy of the proposed metrics on two different manifold learning algorithms when used for motion clustering in video data. Our results show that, in this particular application, the new proposed metrics lead to boosts in performance (at least 7%) when compared to other divergence measures.
RIS
TY - CPAPER TI - A Note on Metric Properties for Some Divergence Measures: The Gaussian Case AU - Karim T. Abou-Moustafa AU - Frank P. Ferrie BT - Proceedings of the Asian Conference on Machine Learning DA - 2012/11/17 ED - Steven C. H. Hoi ED - Wray Buntine ID - pmlr-v25-aboumoustafa12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 25 SP - 1 EP - 15 L1 - http://proceedings.mlr.press/v25/aboumoustafa12/aboumoustafa12.pdf UR - https://proceedings.mlr.press/v25/aboumoustafa12.html AB - Multivariate Gaussian densities are pervasive in pattern recognition and machine learning. A central operation that appears in most of these areas is to measure the difference between two multivariate Gaussians. Unfortunately, traditional measures based on the Kullback-Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics. Next, we show how these metric axioms impact the unfolding process of manifold learning algorithms. Finally, we illustrate the efficacy of the proposed metrics on two different manifold learning algorithms when used for motion clustering in video data. Our results show that, in this particular application, the new proposed metrics lead to boosts in performance (at least 7%) when compared to other divergence measures. ER -
APA
Abou-Moustafa, K.T. & Ferrie, F.P.. (2012). A Note on Metric Properties for Some Divergence Measures: The Gaussian Case. Proceedings of the Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 25:1-15 Available from https://proceedings.mlr.press/v25/aboumoustafa12.html.

Related Material