Efficient GradientBased Inference through Transformations between Bayes Nets and Neural Nets
[edit]
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):17821790, 2014.
Abstract
Hierarchical Bayesian networks and neural networks with stochastic hidden units are commonly perceived as two separate types of models. We show that either of these types of models can often be transformed into an instance of the other, by switching between centered and differentiable noncentered parameterizations of the latent variables. The choice of parameterization greatly influences the efficiency of gradientbased posterior inference; we show that they are often complementary to eachother, we clarify when each parameterization is preferred and show how inference can be made robust. In the noncentered form, a simple Monte Carlo estimator of the marginal likelihood can be used for learning the parameters. Theoretical results are supported by experiments.
Related Material



