Gaussian process nonparametric tensor estimator and its minimax optimality

Heishiro Kanagawa, Taiji Suzuki, Hayato Kobayashi, Nobuyuki Shimizu, Yukihiro Tagami
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1632-1641, 2016.

Abstract

We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-kanagawa16, title = {Gaussian process nonparametric tensor estimator and its minimax optimality}, author = {Kanagawa, Heishiro and Suzuki, Taiji and Kobayashi, Hayato and Shimizu, Nobuyuki and Tagami, Yukihiro}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1632--1641}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/kanagawa16.pdf}, url = {https://proceedings.mlr.press/v48/kanagawa16.html}, abstract = {We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.} }
Endnote
%0 Conference Paper %T Gaussian process nonparametric tensor estimator and its minimax optimality %A Heishiro Kanagawa %A Taiji Suzuki %A Hayato Kobayashi %A Nobuyuki Shimizu %A Yukihiro Tagami %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-kanagawa16 %I PMLR %P 1632--1641 %U https://proceedings.mlr.press/v48/kanagawa16.html %V 48 %X We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.
RIS
TY - CPAPER TI - Gaussian process nonparametric tensor estimator and its minimax optimality AU - Heishiro Kanagawa AU - Taiji Suzuki AU - Hayato Kobayashi AU - Nobuyuki Shimizu AU - Yukihiro Tagami BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-kanagawa16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1632 EP - 1641 L1 - http://proceedings.mlr.press/v48/kanagawa16.pdf UR - https://proceedings.mlr.press/v48/kanagawa16.html AB - We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets. ER -
APA
Kanagawa, H., Suzuki, T., Kobayashi, H., Shimizu, N. & Tagami, Y.. (2016). Gaussian process nonparametric tensor estimator and its minimax optimality. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1632-1641 Available from https://proceedings.mlr.press/v48/kanagawa16.html.

Related Material