Learning Structural Weight Uncertainty for Sequential Decision-Making

Ruiyi Zhang, Chunyuan Li, Changyou Chen, Lawrence Carin
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:1137-1146, 2018.

Abstract

Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the individual NN weights (as often applied), SVGD does not impose prior knowledge that there is often structural information (dependence) among weights. We propose efficient posterior learning of structural weight uncertainty, within an SVGD framework, by employing matrix variate Gaussian priors on NN parameters. We further investigate the learned structural uncertainty in sequential decision-making problems, including contextual bandits and reinforcement learning. Experiments on several synthetic and real datasets indicate the superiority of our model, compared with state-of-the-art methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v84-zhang18d, title = {Learning Structural Weight Uncertainty for Sequential Decision-Making}, author = {Zhang, Ruiyi and Li, Chunyuan and Chen, Changyou and Carin, Lawrence}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1137--1146}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/zhang18d/zhang18d.pdf}, url = {https://proceedings.mlr.press/v84/zhang18d.html}, abstract = {Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the individual NN weights (as often applied), SVGD does not impose prior knowledge that there is often structural information (dependence) among weights. We propose efficient posterior learning of structural weight uncertainty, within an SVGD framework, by employing matrix variate Gaussian priors on NN parameters. We further investigate the learned structural uncertainty in sequential decision-making problems, including contextual bandits and reinforcement learning. Experiments on several synthetic and real datasets indicate the superiority of our model, compared with state-of-the-art methods.} }
Endnote
%0 Conference Paper %T Learning Structural Weight Uncertainty for Sequential Decision-Making %A Ruiyi Zhang %A Chunyuan Li %A Changyou Chen %A Lawrence Carin %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-zhang18d %I PMLR %P 1137--1146 %U https://proceedings.mlr.press/v84/zhang18d.html %V 84 %X Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the individual NN weights (as often applied), SVGD does not impose prior knowledge that there is often structural information (dependence) among weights. We propose efficient posterior learning of structural weight uncertainty, within an SVGD framework, by employing matrix variate Gaussian priors on NN parameters. We further investigate the learned structural uncertainty in sequential decision-making problems, including contextual bandits and reinforcement learning. Experiments on several synthetic and real datasets indicate the superiority of our model, compared with state-of-the-art methods.
APA
Zhang, R., Li, C., Chen, C. & Carin, L.. (2018). Learning Structural Weight Uncertainty for Sequential Decision-Making. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:1137-1146 Available from https://proceedings.mlr.press/v84/zhang18d.html.

Related Material