Efficient Training of LDA on a GPU by Mean-for-Mode Estimation


Jean-Baptiste Tristan, Joseph Tassarotti, Guy Steele ;
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:59-68, 2015.


We introduce Mean-for-Mode estimation, a variant of an uncollapsed Gibbs sampler that we use to train LDA on a GPU. The algorithm combines benefits of both uncollapsed and collapsed Gibbs samplers. Like a collapsed Gibbs sampler — and unlike an uncollapsed Gibbs sampler — it has good statistical performance, and can use sampling complexity reduction techniques such as sparsity. Meanwhile, like an uncollapsed Gibbs sampler — and unlike a collapsed Gibbs sampler — it is embarrassingly parallel, and can use approximate counters.

Related Material