GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning

Idan Achituve, Aviv Navon, Yochai Yemini, Gal Chechik, Ethan Fetaya
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:54-65, 2021.

Abstract

Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the P{ó}lya-Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-achituve21a, title = {GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning}, author = {Achituve, Idan and Navon, Aviv and Yemini, Yochai and Chechik, Gal and Fetaya, Ethan}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {54--65}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/achituve21a/achituve21a.pdf}, url = {https://proceedings.mlr.press/v139/achituve21a.html}, abstract = {Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the P{ó}lya-Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.} }
Endnote
%0 Conference Paper %T GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning %A Idan Achituve %A Aviv Navon %A Yochai Yemini %A Gal Chechik %A Ethan Fetaya %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-achituve21a %I PMLR %P 54--65 %U https://proceedings.mlr.press/v139/achituve21a.html %V 139 %X Gaussian processes (GPs) are non-parametric, flexible, models that work well in many tasks. Combining GPs with deep learning methods via deep kernel learning (DKL) is especially compelling due to the strong representational power induced by the network. However, inference in GPs, whether with or without DKL, can be computationally challenging on large datasets. Here, we propose GP-Tree, a novel method for multi-class classification with Gaussian processes and DKL. We develop a tree-based hierarchical model in which each internal node of the tree fits a GP to the data using the P{ó}lya-Gamma augmentation scheme. As a result, our method scales well with both the number of classes and data size. We demonstrate the effectiveness of our method against other Gaussian process training baselines, and we show how our general GP approach achieves improved accuracy on standard incremental few-shot learning benchmarks.
APA
Achituve, I., Navon, A., Yemini, Y., Chechik, G. & Fetaya, E.. (2021). GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:54-65 Available from https://proceedings.mlr.press/v139/achituve21a.html.

Related Material