Non-parametric Inference Adaptive to Intrinsic Dimension

Khashayar Khosravi, Greg Lewis, Vasilis Syrgkanis
Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:373-389, 2022.

Abstract

We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v177-khosravi22a, title = {Non-parametric Inference Adaptive to Intrinsic Dimension}, author = {Khosravi, Khashayar and Lewis, Greg and Syrgkanis, Vasilis}, booktitle = {Proceedings of the First Conference on Causal Learning and Reasoning}, pages = {373--389}, year = {2022}, editor = {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun}, volume = {177}, series = {Proceedings of Machine Learning Research}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v177/khosravi22a/khosravi22a.pdf}, url = {https://proceedings.mlr.press/v177/khosravi22a.html}, abstract = {We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.} }
Endnote
%0 Conference Paper %T Non-parametric Inference Adaptive to Intrinsic Dimension %A Khashayar Khosravi %A Greg Lewis %A Vasilis Syrgkanis %B Proceedings of the First Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2022 %E Bernhard Schölkopf %E Caroline Uhler %E Kun Zhang %F pmlr-v177-khosravi22a %I PMLR %P 373--389 %U https://proceedings.mlr.press/v177/khosravi22a.html %V 177 %X We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.
APA
Khosravi, K., Lewis, G. & Syrgkanis, V.. (2022). Non-parametric Inference Adaptive to Intrinsic Dimension. Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:373-389 Available from https://proceedings.mlr.press/v177/khosravi22a.html.

Related Material