Deep Kernel Regression with Finite Learnable Kernels

Chunlin Ji, Yuhao Fu
Proceedings of the 15th Asian Conference on Machine Learning, PMLR 222:550-565, 2024.

Abstract

In this work, we study kernel regression that integrates with a modern deep neural network (DNN). The DNN projects the input into an embedding space, meanwhile a set of representative points is constructed in this embedding space. We build the regression using a finite set of kernels defined on the embedding space, where the DNN weights, the regression coefficients, and kernel hyperparameters are all learnable. We extend the model by introducing a location attention strategy and the multiple kernel technique. We provide effective ways to obtain representative points. The proposed model can be trained with an end-to-end learning algorithm with simple implementation. Simulation studies show that the proposed deep kernel regression is well scalable to large datasets and comparable to or superior to recent deep kernel models in various regression problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v222-ji24b, title = {Deep Kernel Regression with Finite Learnable Kernels}, author = {Ji, Chunlin and Fu, Yuhao}, booktitle = {Proceedings of the 15th Asian Conference on Machine Learning}, pages = {550--565}, year = {2024}, editor = {Yanıkoğlu, Berrin and Buntine, Wray}, volume = {222}, series = {Proceedings of Machine Learning Research}, month = {11--14 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v222/ji24b/ji24b.pdf}, url = {https://proceedings.mlr.press/v222/ji24b.html}, abstract = {In this work, we study kernel regression that integrates with a modern deep neural network (DNN). The DNN projects the input into an embedding space, meanwhile a set of representative points is constructed in this embedding space. We build the regression using a finite set of kernels defined on the embedding space, where the DNN weights, the regression coefficients, and kernel hyperparameters are all learnable. We extend the model by introducing a location attention strategy and the multiple kernel technique. We provide effective ways to obtain representative points. The proposed model can be trained with an end-to-end learning algorithm with simple implementation. Simulation studies show that the proposed deep kernel regression is well scalable to large datasets and comparable to or superior to recent deep kernel models in various regression problems.} }
Endnote
%0 Conference Paper %T Deep Kernel Regression with Finite Learnable Kernels %A Chunlin Ji %A Yuhao Fu %B Proceedings of the 15th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Berrin Yanıkoğlu %E Wray Buntine %F pmlr-v222-ji24b %I PMLR %P 550--565 %U https://proceedings.mlr.press/v222/ji24b.html %V 222 %X In this work, we study kernel regression that integrates with a modern deep neural network (DNN). The DNN projects the input into an embedding space, meanwhile a set of representative points is constructed in this embedding space. We build the regression using a finite set of kernels defined on the embedding space, where the DNN weights, the regression coefficients, and kernel hyperparameters are all learnable. We extend the model by introducing a location attention strategy and the multiple kernel technique. We provide effective ways to obtain representative points. The proposed model can be trained with an end-to-end learning algorithm with simple implementation. Simulation studies show that the proposed deep kernel regression is well scalable to large datasets and comparable to or superior to recent deep kernel models in various regression problems.
APA
Ji, C. & Fu, Y.. (2024). Deep Kernel Regression with Finite Learnable Kernels. Proceedings of the 15th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 222:550-565 Available from https://proceedings.mlr.press/v222/ji24b.html.

Related Material