From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation

Wenyuan Zhao, Haoyuan Chen, Tie Liu, Rui Tuo, Chao Tian
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:4231-4239, 2025.

Abstract

With the strengths of both deep learning and kernel methods like Gaussian Processes (GPs), Deep Kernel Learning (DKL) has gained considerable attention in recent years. From the computational perspective, however, DKL becomes challenging when the input dimension of the GP layer is high. To address this challenge, we propose the Deep Additive Kernel (DAK) model, which incorporates i) an additive structure for the last-layer GP; and ii) induced prior approximation for each GP unit. This naturally leads to a last-layer Bayesian neural network (BNN) architecture. The proposed method enjoys the interpretability of DKL as well as the computational advantages of BNN. Empirical results show that the proposed approach outperforms state-of-the-art DKL methods in both regression and classification tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-zhao25d, title = {From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation}, author = {Zhao, Wenyuan and Chen, Haoyuan and Liu, Tie and Tuo, Rui and Tian, Chao}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {4231--4239}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/zhao25d/zhao25d.pdf}, url = {https://proceedings.mlr.press/v258/zhao25d.html}, abstract = {With the strengths of both deep learning and kernel methods like Gaussian Processes (GPs), Deep Kernel Learning (DKL) has gained considerable attention in recent years. From the computational perspective, however, DKL becomes challenging when the input dimension of the GP layer is high. To address this challenge, we propose the Deep Additive Kernel (DAK) model, which incorporates i) an additive structure for the last-layer GP; and ii) induced prior approximation for each GP unit. This naturally leads to a last-layer Bayesian neural network (BNN) architecture. The proposed method enjoys the interpretability of DKL as well as the computational advantages of BNN. Empirical results show that the proposed approach outperforms state-of-the-art DKL methods in both regression and classification tasks.} }
Endnote
%0 Conference Paper %T From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation %A Wenyuan Zhao %A Haoyuan Chen %A Tie Liu %A Rui Tuo %A Chao Tian %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-zhao25d %I PMLR %P 4231--4239 %U https://proceedings.mlr.press/v258/zhao25d.html %V 258 %X With the strengths of both deep learning and kernel methods like Gaussian Processes (GPs), Deep Kernel Learning (DKL) has gained considerable attention in recent years. From the computational perspective, however, DKL becomes challenging when the input dimension of the GP layer is high. To address this challenge, we propose the Deep Additive Kernel (DAK) model, which incorporates i) an additive structure for the last-layer GP; and ii) induced prior approximation for each GP unit. This naturally leads to a last-layer Bayesian neural network (BNN) architecture. The proposed method enjoys the interpretability of DKL as well as the computational advantages of BNN. Empirical results show that the proposed approach outperforms state-of-the-art DKL methods in both regression and classification tasks.
APA
Zhao, W., Chen, H., Liu, T., Tuo, R. & Tian, C.. (2025). From Deep Additive Kernel Learning to Last-Layer Bayesian Neural Networks via Induced Prior Approximation. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:4231-4239 Available from https://proceedings.mlr.press/v258/zhao25d.html.

Related Material