PAC-Bayes Analysis Of Maximum Entropy Classification


John Shawe-Taylor, David Hardoon ;
Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics, PMLR 5:480-487, 2009.


We extend and apply the PAC-Bayes theorem to the analysis of maximum entropy learning by considering maximum entropy classification. The theory introduces a multiple sampling technique that controls an effective margin of the bound. We further develop a dual implementation of the convex optimisation that optimises the bound. This algorithm is tested on some simple datasets and the value of the bound compared with the test error.

Related Material