LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood

Piotr Tempczyk, Rafał Michaluk, Lukasz Garncarek, Przemysław Spurek, Jacek Tabor, Adam Golinski
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:21205-21231, 2022.

Abstract

Most of the existing methods for estimating the local intrinsic dimension of a data distribution do not scale well to high dimensional data. Many of them rely on a non-parametric nearest neighbours approach which suffers from the curse of dimensionality. We attempt to address that challenge by proposing a novel approach to the problem: Local Intrinsic Dimension estimation using approximate Likelihood (LIDL). Our method relies on an arbitrary density estimation method as its subroutine, and hence tries to sidestep the dimensionality challenge by making use of the recent progress in parametric neural methods for likelihood estimation. We carefully investigate the empirical properties of the proposed method, compare them with our theoretical predictions, show that LIDL yields competitive results on the standard benchmarks for this problem, and that it scales to thousands of dimensions. What is more, we anticipate this approach to improve further with the continuing advances in the density estimation literature.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-tempczyk22a, title = {{LIDL}: Local Intrinsic Dimension Estimation Using Approximate Likelihood}, author = {Tempczyk, Piotr and Michaluk, Rafa{\l} and Garncarek, Lukasz and Spurek, Przemys{\l}aw and Tabor, Jacek and Golinski, Adam}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {21205--21231}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/tempczyk22a/tempczyk22a.pdf}, url = {https://proceedings.mlr.press/v162/tempczyk22a.html}, abstract = {Most of the existing methods for estimating the local intrinsic dimension of a data distribution do not scale well to high dimensional data. Many of them rely on a non-parametric nearest neighbours approach which suffers from the curse of dimensionality. We attempt to address that challenge by proposing a novel approach to the problem: Local Intrinsic Dimension estimation using approximate Likelihood (LIDL). Our method relies on an arbitrary density estimation method as its subroutine, and hence tries to sidestep the dimensionality challenge by making use of the recent progress in parametric neural methods for likelihood estimation. We carefully investigate the empirical properties of the proposed method, compare them with our theoretical predictions, show that LIDL yields competitive results on the standard benchmarks for this problem, and that it scales to thousands of dimensions. What is more, we anticipate this approach to improve further with the continuing advances in the density estimation literature.} }
Endnote
%0 Conference Paper %T LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood %A Piotr Tempczyk %A Rafał Michaluk %A Lukasz Garncarek %A Przemysław Spurek %A Jacek Tabor %A Adam Golinski %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-tempczyk22a %I PMLR %P 21205--21231 %U https://proceedings.mlr.press/v162/tempczyk22a.html %V 162 %X Most of the existing methods for estimating the local intrinsic dimension of a data distribution do not scale well to high dimensional data. Many of them rely on a non-parametric nearest neighbours approach which suffers from the curse of dimensionality. We attempt to address that challenge by proposing a novel approach to the problem: Local Intrinsic Dimension estimation using approximate Likelihood (LIDL). Our method relies on an arbitrary density estimation method as its subroutine, and hence tries to sidestep the dimensionality challenge by making use of the recent progress in parametric neural methods for likelihood estimation. We carefully investigate the empirical properties of the proposed method, compare them with our theoretical predictions, show that LIDL yields competitive results on the standard benchmarks for this problem, and that it scales to thousands of dimensions. What is more, we anticipate this approach to improve further with the continuing advances in the density estimation literature.
APA
Tempczyk, P., Michaluk, R., Garncarek, L., Spurek, P., Tabor, J. & Golinski, A.. (2022). LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:21205-21231 Available from https://proceedings.mlr.press/v162/tempczyk22a.html.

Related Material