A Bit More Bayesian: Domain-Invariant Learning with Uncertainty

Zehao Xiao, Jiayi Shen, Xiantong Zhen, Ling Shao, Cees Snoek
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:11351-11361, 2021.

Abstract

Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-xiao21a, title = {A Bit More Bayesian: Domain-Invariant Learning with Uncertainty}, author = {Xiao, Zehao and Shen, Jiayi and Zhen, Xiantong and Shao, Ling and Snoek, Cees}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {11351--11361}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/xiao21a/xiao21a.pdf}, url = {https://proceedings.mlr.press/v139/xiao21a.html}, abstract = {Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.} }
Endnote
%0 Conference Paper %T A Bit More Bayesian: Domain-Invariant Learning with Uncertainty %A Zehao Xiao %A Jiayi Shen %A Xiantong Zhen %A Ling Shao %A Cees Snoek %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-xiao21a %I PMLR %P 11351--11361 %U https://proceedings.mlr.press/v139/xiao21a.html %V 139 %X Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.
APA
Xiao, Z., Shen, J., Zhen, X., Shao, L. & Snoek, C.. (2021). A Bit More Bayesian: Domain-Invariant Learning with Uncertainty. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:11351-11361 Available from https://proceedings.mlr.press/v139/xiao21a.html.

Related Material