Gaussian process nonparametric tensor estimator and its minimax optimality

Heishiro Kanagawa, Taiji Suzuki, Hayato Kobayashi, Nobuyuki Shimizu, Yukihiro Tagami
; Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1632-1641, 2016.

Abstract

We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.

Cite this Paper


BibTeX
@InProceedings{pmlr-v48-kanagawa16, title = {Gaussian process nonparametric tensor estimator and its minimax optimality}, author = {Heishiro Kanagawa and Taiji Suzuki and Hayato Kobayashi and Nobuyuki Shimizu and Yukihiro Tagami}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1632--1641}, year = {2016}, editor = {Maria Florina Balcan and Kilian Q. Weinberger}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/kanagawa16.pdf}, url = {http://proceedings.mlr.press/v48/kanagawa16.html}, abstract = {We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.} }
Endnote
%0 Conference Paper %T Gaussian process nonparametric tensor estimator and its minimax optimality %A Heishiro Kanagawa %A Taiji Suzuki %A Hayato Kobayashi %A Nobuyuki Shimizu %A Yukihiro Tagami %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-kanagawa16 %I PMLR %J Proceedings of Machine Learning Research %P 1632--1641 %U http://proceedings.mlr.press %V 48 %W PMLR %X We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets.
RIS
TY - CPAPER TI - Gaussian process nonparametric tensor estimator and its minimax optimality AU - Heishiro Kanagawa AU - Taiji Suzuki AU - Hayato Kobayashi AU - Nobuyuki Shimizu AU - Yukihiro Tagami BT - Proceedings of The 33rd International Conference on Machine Learning PY - 2016/06/11 DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-kanagawa16 PB - PMLR SP - 1632 DP - PMLR EP - 1641 L1 - http://proceedings.mlr.press/v48/kanagawa16.pdf UR - http://proceedings.mlr.press/v48/kanagawa16.html AB - We investigate the statistical efficiency of a nonparametric Gaussian process method for a nonlinear tensor estimation problem. Low-rank tensor estimation has been used as a method to learn higher order relations among several data sources in a wide range of applications, such as multi-task learning, recommendation systems, and spatiotemporal analysis. We consider a general setting where a common linear tensor learning is extended to a nonlinear learning problem in reproducing kernel Hilbert space and propose a nonparametric Bayesian method based on the Gaussian process method. We prove its statistical convergence rate without assuming any strong convexity, such as restricted strong convexity. Remarkably, it is shown that our convergence rate achieves the minimax optimal rate. We apply our proposed method to multi-task learning and show that our method significantly outperforms existing methods through numerical experiments on real-world data sets. ER -
APA
Kanagawa, H., Suzuki, T., Kobayashi, H., Shimizu, N. & Tagami, Y.. (2016). Gaussian process nonparametric tensor estimator and its minimax optimality. Proceedings of The 33rd International Conference on Machine Learning, in PMLR 48:1632-1641

Related Material