Statistical Estimation from Dependent Data

Vardis Kandiros, Yuval Dagan, Nishanth Dikkala, Surbhi Goel, Constantinos Daskalakis
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5269-5278, 2021.

Abstract

We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioning on their feature vectors, but dependent, capturing settings where e.g. these observations are collected on a spatial domain, a temporal domain, or a social network, which induce dependencies. We model these dependencies in the language of Markov Random Fields and, importantly, allow these dependencies to be substantial, i.e. do not assume that the Markov Random Field capturing these dependencies is in high temperature. As our main contribution we provide algorithms and statistically efficient estimation rates for this model, giving several instantiations of our bounds in logistic regression, sparse logistic regression, and neural network regression settings with dependent data. Our estimation guarantees follow from novel results for estimating the parameters (i.e. external fields and interaction strengths) of Ising models from a single sample.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-kandiros21a, title = {Statistical Estimation from Dependent Data}, author = {Kandiros, Vardis and Dagan, Yuval and Dikkala, Nishanth and Goel, Surbhi and Daskalakis, Constantinos}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {5269--5278}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/kandiros21a/kandiros21a.pdf}, url = {https://proceedings.mlr.press/v139/kandiros21a.html}, abstract = {We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioning on their feature vectors, but dependent, capturing settings where e.g. these observations are collected on a spatial domain, a temporal domain, or a social network, which induce dependencies. We model these dependencies in the language of Markov Random Fields and, importantly, allow these dependencies to be substantial, i.e. do not assume that the Markov Random Field capturing these dependencies is in high temperature. As our main contribution we provide algorithms and statistically efficient estimation rates for this model, giving several instantiations of our bounds in logistic regression, sparse logistic regression, and neural network regression settings with dependent data. Our estimation guarantees follow from novel results for estimating the parameters (i.e. external fields and interaction strengths) of Ising models from a single sample.} }
Endnote
%0 Conference Paper %T Statistical Estimation from Dependent Data %A Vardis Kandiros %A Yuval Dagan %A Nishanth Dikkala %A Surbhi Goel %A Constantinos Daskalakis %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-kandiros21a %I PMLR %P 5269--5278 %U https://proceedings.mlr.press/v139/kandiros21a.html %V 139 %X We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioning on their feature vectors, but dependent, capturing settings where e.g. these observations are collected on a spatial domain, a temporal domain, or a social network, which induce dependencies. We model these dependencies in the language of Markov Random Fields and, importantly, allow these dependencies to be substantial, i.e. do not assume that the Markov Random Field capturing these dependencies is in high temperature. As our main contribution we provide algorithms and statistically efficient estimation rates for this model, giving several instantiations of our bounds in logistic regression, sparse logistic regression, and neural network regression settings with dependent data. Our estimation guarantees follow from novel results for estimating the parameters (i.e. external fields and interaction strengths) of Ising models from a single sample.
APA
Kandiros, V., Dagan, Y., Dikkala, N., Goel, S. & Daskalakis, C.. (2021). Statistical Estimation from Dependent Data. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:5269-5278 Available from https://proceedings.mlr.press/v139/kandiros21a.html.

Related Material