Constrained Bayesian Inference for Low Rank Multitask Learning

Oluwasanmi Koyejo, Joydeep Ghosh
Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, PMLR R11:135-144, 2013.

Abstract

We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the con- straint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply the proposed constrained Bayesian inference approach to mul- titask learning subject to rank constraints on the weight matrix. Further, constrained parameter estimation is applied to recover the sparse con- ditional independence structure encoded by prior precision matrices. Our approach is motivated by reverse inference for high dimensional func- tional neuroimaging, a domain where the high dimensionality and small number of examples re- quires the use of constraints to ensure meaning- ful and effective models. For this application, we propose a model that jointly learns a weight ma- trix and the prior inverse covariance structure be- tween different tasks. We present experimental validation showing that the proposed approach outperforms strong baseline models in terms of predictive performance and structure recovery.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR11-koyejo13a, title = {Constrained {B}ayesian Inference for Low Rank Multitask Learning}, author = {Koyejo, Oluwasanmi and Ghosh, Joydeep}, booktitle = {Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence}, pages = {135--144}, year = {2013}, editor = {Nicholson, Ann and Smyth, Padhraic}, volume = {R11}, series = {Proceedings of Machine Learning Research}, month = {12--14 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r11/main/assets/koyejo13a/koyejo13a.pdf}, url = {https://proceedings.mlr.press/r11/koyejo13a.html}, abstract = {We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the con- straint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply the proposed constrained Bayesian inference approach to mul- titask learning subject to rank constraints on the weight matrix. Further, constrained parameter estimation is applied to recover the sparse con- ditional independence structure encoded by prior precision matrices. Our approach is motivated by reverse inference for high dimensional func- tional neuroimaging, a domain where the high dimensionality and small number of examples re- quires the use of constraints to ensure meaning- ful and effective models. For this application, we propose a model that jointly learns a weight ma- trix and the prior inverse covariance structure be- tween different tasks. We present experimental validation showing that the proposed approach outperforms strong baseline models in terms of predictive performance and structure recovery.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Constrained Bayesian Inference for Low Rank Multitask Learning %A Oluwasanmi Koyejo %A Joydeep Ghosh %B Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2013 %E Ann Nicholson %E Padhraic Smyth %F pmlr-vR11-koyejo13a %I PMLR %P 135--144 %U https://proceedings.mlr.press/r11/koyejo13a.html %V R11 %X We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the con- straint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply the proposed constrained Bayesian inference approach to mul- titask learning subject to rank constraints on the weight matrix. Further, constrained parameter estimation is applied to recover the sparse con- ditional independence structure encoded by prior precision matrices. Our approach is motivated by reverse inference for high dimensional func- tional neuroimaging, a domain where the high dimensionality and small number of examples re- quires the use of constraints to ensure meaning- ful and effective models. For this application, we propose a model that jointly learns a weight ma- trix and the prior inverse covariance structure be- tween different tasks. We present experimental validation showing that the proposed approach outperforms strong baseline models in terms of predictive performance and structure recovery. %Z Reissued by PMLR on 04 October 2026.
APA
Koyejo, O. & Ghosh, J.. (2013). Constrained Bayesian Inference for Low Rank Multitask Learning. Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R11:135-144 Available from https://proceedings.mlr.press/r11/koyejo13a.html. Reissued by PMLR on 04 October 2026.

Related Material