Robust RegBayes: Selectively Incorporating First-Order Logic Domain Knowledge into Bayesian Models


Shike Mei, Jun Zhu, Jerry Zhu ;
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(1):253-261, 2014.


Much research in Bayesian modeling has been done to elicit a prior distribution that incorporates domain knowledge. We present a novel and more direct approach by imposing First-Order Logic (FOL) rules on the posterior distribution. Our approach unifies FOL and Bayesian modeling under the regularized Bayesian framework. In addition, our approach automatically estimates the uncertainty of FOL rules when they are produced by humans, so that reliable rules are incorporated while unreliable ones are ignored. We apply our approach to latent topic modeling tasks and demonstrate that by combining FOL knowledge and Bayesian modeling, we both improve the task performance and discover more structured latent representations in unsupervised and supervised learning.

Related Material