Non-parametric Inference Adaptive to Intrinsic Dimension

Khashayar Khosravi; Greg Lewis; Vasilis Syrgkanis

Non-parametric Inference Adaptive to Intrinsic Dimension

Khashayar Khosravi, Greg Lewis, Vasilis Syrgkanis

Proceedings of the First Conference on Causal Learning and Reasoning, PMLR 177:373-389, 2022.

Abstract

We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension

$D$ of the conditioning variable is larger than the sample size

$n$ , estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension

$d$ , as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the

$k$ -nearest neighbors (

$k$ -NN)

$Z$ -estimator. We show that if the intrinsic dimension of the covariate distribution is equal to

$d$ , then the finite sample estimation error of our estimator is of order

$n^{-1/(d+2)}$ and our estimate is

$n^{1/(d+2)}$ -asymptotically normal, irrespective of

$D$ . The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension

$d$ . We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.

Cite this Paper

BibTeX


@InProceedings{pmlr-v177-khosravi22a,
  title = 	 {Non-parametric Inference Adaptive to Intrinsic Dimension},
  author =       {Khosravi, Khashayar and Lewis, Greg and Syrgkanis, Vasilis},
  booktitle = 	 {Proceedings of the First Conference on Causal Learning and Reasoning},
  pages = 	 {373--389},
  year = 	 {2022},
  editor = 	 {Schölkopf, Bernhard and Uhler, Caroline and Zhang, Kun},
  volume = 	 {177},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {11--13 Apr},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v177/khosravi22a/khosravi22a.pdf},
  url = 	 {https://proceedings.mlr.press/v177/khosravi22a.html},
  abstract = 	 {We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.}
}

Endnote

%0 Conference Paper
%T Non-parametric Inference Adaptive to Intrinsic Dimension
%A Khashayar Khosravi
%A Greg Lewis
%A Vasilis Syrgkanis
%B Proceedings of the First Conference on Causal Learning and Reasoning
%C Proceedings of Machine Learning Research
%D 2022
%E Bernhard Schölkopf
%E Caroline Uhler
%E Kun Zhang	
%F pmlr-v177-khosravi22a
%I PMLR
%P 373--389
%U https://proceedings.mlr.press/v177/khosravi22a.html
%V 177
%X We consider non-parametric estimation and inference of conditional moment models in high dimensions. We show that even when the dimension $D$ of the conditioning variable is larger than the sample size $n$, estimation and inference is feasible as long as the distribution of the conditioning variable has small intrinsic dimension $d$, as measured by locally low doubling measures. Our estimation is based on a sub-sampled ensemble of the $k$-nearest neighbors ($k$-NN) $Z$-estimator. We show that if the intrinsic dimension of the covariate distribution is equal to $d$, then the finite sample estimation error of our estimator is of order $n^{-1/(d+2)}$ and our estimate is $n^{1/(d+2)}$-asymptotically normal, irrespective of $D$. The sub-sampling size required for achieving these results depends on the unknown intrinsic dimension $d$. We propose an adaptive data-driven approach for choosing this parameter and prove that it achieves the desired rates. We discuss extensions and applications to heterogeneous treatment effect estimation.

APA


Khosravi, K., Lewis, G. & Syrgkanis, V.. (2022). Non-parametric Inference Adaptive to Intrinsic Dimension. Proceedings of the First Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 177:373-389 Available from https://proceedings.mlr.press/v177/khosravi22a.html.

Related Material

Download PDF