Spectral Dimensionality Reduction via Maximum Entropy

Neil Lawrence
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:51-59, 2011.

Abstract

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We develop new algorithms that directly maximize the likelihood and show that these new algorithms are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.

Cite this Paper

BibTeX
@InProceedings{pmlr-v15-lawrence11a, title = {Spectral Dimensionality Reduction via Maximum Entropy}, author = {Lawrence, Neil}, booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, pages = {51--59}, year = {2011}, editor = {Gordon, Geoffrey and Dunson, David and Dudík, Miroslav}, volume = {15}, series = {Proceedings of Machine Learning Research}, address = {Fort Lauderdale, FL, USA}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v15/lawrence11a/lawrence11a.pdf}, url = {https://proceedings.mlr.press/v15/lawrence11a.html}, abstract = {We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We develop new algorithms that directly maximize the likelihood and show that these new algorithms are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. } }
Endnote
%0 Conference Paper %T Spectral Dimensionality Reduction via Maximum Entropy %A Neil Lawrence %B Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2011 %E Geoffrey Gordon %E David Dunson %E Miroslav Dudík %F pmlr-v15-lawrence11a %I PMLR %P 51--59 %U https://proceedings.mlr.press/v15/lawrence11a.html %V 15 %X We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We develop new algorithms that directly maximize the likelihood and show that these new algorithms are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set.
RIS
TY - CPAPER TI - Spectral Dimensionality Reduction via Maximum Entropy AU - Neil Lawrence BT - Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics DA - 2011/06/14 ED - Geoffrey Gordon ED - David Dunson ED - Miroslav Dudík ID - pmlr-v15-lawrence11a PB - PMLR DP - Proceedings of Machine Learning Research VL - 15 SP - 51 EP - 59 L1 - http://proceedings.mlr.press/v15/lawrence11a/lawrence11a.pdf UR - https://proceedings.mlr.press/v15/lawrence11a.html AB - We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting probabilistic models are based on GRFs. The resulting model is a nonlinear generalization of principal component analysis. We show that parameter fitting in the locally linear embedding is approximate maximum likelihood in these models. We develop new algorithms that directly maximize the likelihood and show that these new algorithms are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. ER -
APA
Lawrence, N.. (2011). Spectral Dimensionality Reduction via Maximum Entropy. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 15:51-59 Available from https://proceedings.mlr.press/v15/lawrence11a.html.

Related Material