Kernel Logistic Regression Approximation of an Understandable ReLU Neural Network

Marie Guyomard, Susana Barbosa, Lionel Fillatre
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:12268-12291, 2023.

Abstract

This paper proposes an understandable neural network whose score function is modeled as an additive sum of univariate spline functions. It extends usual understandable models like generative additive models, spline-based models, and neural additive models. It is shown that this neural network can be approximated by a logistic regression whose inputs are obtained with a non-linear preprocessing of input data. This preprocessing depends on the neural network initialization but this paper establishes that it can be replaced by a non random kernel-based preprocessing that no longer depends on the initialization. Hence, the convergence of the training process is guaranteed and the solution is unique for a given training dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-guyomard23a, title = {Kernel Logistic Regression Approximation of an Understandable {R}e{LU} Neural Network}, author = {Guyomard, Marie and Barbosa, Susana and Fillatre, Lionel}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {12268--12291}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/guyomard23a/guyomard23a.pdf}, url = {https://proceedings.mlr.press/v202/guyomard23a.html}, abstract = {This paper proposes an understandable neural network whose score function is modeled as an additive sum of univariate spline functions. It extends usual understandable models like generative additive models, spline-based models, and neural additive models. It is shown that this neural network can be approximated by a logistic regression whose inputs are obtained with a non-linear preprocessing of input data. This preprocessing depends on the neural network initialization but this paper establishes that it can be replaced by a non random kernel-based preprocessing that no longer depends on the initialization. Hence, the convergence of the training process is guaranteed and the solution is unique for a given training dataset.} }
Endnote
%0 Conference Paper %T Kernel Logistic Regression Approximation of an Understandable ReLU Neural Network %A Marie Guyomard %A Susana Barbosa %A Lionel Fillatre %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-guyomard23a %I PMLR %P 12268--12291 %U https://proceedings.mlr.press/v202/guyomard23a.html %V 202 %X This paper proposes an understandable neural network whose score function is modeled as an additive sum of univariate spline functions. It extends usual understandable models like generative additive models, spline-based models, and neural additive models. It is shown that this neural network can be approximated by a logistic regression whose inputs are obtained with a non-linear preprocessing of input data. This preprocessing depends on the neural network initialization but this paper establishes that it can be replaced by a non random kernel-based preprocessing that no longer depends on the initialization. Hence, the convergence of the training process is guaranteed and the solution is unique for a given training dataset.
APA
Guyomard, M., Barbosa, S. & Fillatre, L.. (2023). Kernel Logistic Regression Approximation of an Understandable ReLU Neural Network. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:12268-12291 Available from https://proceedings.mlr.press/v202/guyomard23a.html.

Related Material