Linear predictor on linearly-generated data with missing values: non consistency and solutions

Marine Le Morvan, Nicolas Prost, Julie Josse, Erwan Scornet, Gael Varoquaux
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:3165-3174, 2020.

Abstract

We consider building predictors when the data have missing values. We study the seemingly-simple case where the target to predict is a linear function of the fully observed data and we show that, in the presence of missing values, the optimal predictor is not linear in general. In the particular Gaussian case, it can be written as a linear function of multiway interactions between the observed data and the various missing value indicators. Due to its intrinsic complexity, we study a simple approximation and prove generalization bounds with finite samples, highlighting regimes for which each method performs best. We then show that multilayer perceptrons with ReLU activation functions can be consistent, and can explore good trade-offs between the true model and approximations. Our study highlights the interesting family of models that are beneficial to fit with missing values depending on the amount of data available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-morvan20a, title = {Linear predictor on linearly-generated data with missing values: non consistency and solutions}, author = {Morvan, Marine Le and Prost, Nicolas and Josse, Julie and Scornet, Erwan and Varoquaux, Gael}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {3165--3174}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/morvan20a/morvan20a.pdf}, url = {https://proceedings.mlr.press/v108/morvan20a.html}, abstract = {We consider building predictors when the data have missing values. We study the seemingly-simple case where the target to predict is a linear function of the fully observed data and we show that, in the presence of missing values, the optimal predictor is not linear in general. In the particular Gaussian case, it can be written as a linear function of multiway interactions between the observed data and the various missing value indicators. Due to its intrinsic complexity, we study a simple approximation and prove generalization bounds with finite samples, highlighting regimes for which each method performs best. We then show that multilayer perceptrons with ReLU activation functions can be consistent, and can explore good trade-offs between the true model and approximations. Our study highlights the interesting family of models that are beneficial to fit with missing values depending on the amount of data available.} }
Endnote
%0 Conference Paper %T Linear predictor on linearly-generated data with missing values: non consistency and solutions %A Marine Le Morvan %A Nicolas Prost %A Julie Josse %A Erwan Scornet %A Gael Varoquaux %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-morvan20a %I PMLR %P 3165--3174 %U https://proceedings.mlr.press/v108/morvan20a.html %V 108 %X We consider building predictors when the data have missing values. We study the seemingly-simple case where the target to predict is a linear function of the fully observed data and we show that, in the presence of missing values, the optimal predictor is not linear in general. In the particular Gaussian case, it can be written as a linear function of multiway interactions between the observed data and the various missing value indicators. Due to its intrinsic complexity, we study a simple approximation and prove generalization bounds with finite samples, highlighting regimes for which each method performs best. We then show that multilayer perceptrons with ReLU activation functions can be consistent, and can explore good trade-offs between the true model and approximations. Our study highlights the interesting family of models that are beneficial to fit with missing values depending on the amount of data available.
APA
Morvan, M.L., Prost, N., Josse, J., Scornet, E. & Varoquaux, G.. (2020). Linear predictor on linearly-generated data with missing values: non consistency and solutions. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:3165-3174 Available from https://proceedings.mlr.press/v108/morvan20a.html.

Related Material