Complementary Sum Sampling for Likelihood Approximation in Large Scale Classification

Aleksandar Botev, Bowen Zheng, David Barber
Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR 54:1030-1038, 2017.

Abstract

We consider training probabilistic classifiers in the case that the number of classes is too large to perform exact normalisation over all classes. We show that the source of high variance in standard sampling approximations is due to simply not including the correct class of the datapoint into the approximation. To account for this we explicitly sum over a subset of classes and sample the remaining. We show that this simple approach is competitive with recently introduced non likelihood-based approximations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v54-botev17a, title = {{Complementary Sum Sampling for Likelihood Approximation in Large Scale Classification}}, author = {Botev, Aleksandar and Zheng, Bowen and Barber, David}, booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence and Statistics}, pages = {1030--1038}, year = {2017}, editor = {Singh, Aarti and Zhu, Jerry}, volume = {54}, series = {Proceedings of Machine Learning Research}, month = {20--22 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v54/botev17a/botev17a.pdf}, url = {https://proceedings.mlr.press/v54/botev17a.html}, abstract = {We consider training probabilistic classifiers in the case that the number of classes is too large to perform exact normalisation over all classes. We show that the source of high variance in standard sampling approximations is due to simply not including the correct class of the datapoint into the approximation. To account for this we explicitly sum over a subset of classes and sample the remaining. We show that this simple approach is competitive with recently introduced non likelihood-based approximations. } }
Endnote
%0 Conference Paper %T Complementary Sum Sampling for Likelihood Approximation in Large Scale Classification %A Aleksandar Botev %A Bowen Zheng %A David Barber %B Proceedings of the 20th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2017 %E Aarti Singh %E Jerry Zhu %F pmlr-v54-botev17a %I PMLR %P 1030--1038 %U https://proceedings.mlr.press/v54/botev17a.html %V 54 %X We consider training probabilistic classifiers in the case that the number of classes is too large to perform exact normalisation over all classes. We show that the source of high variance in standard sampling approximations is due to simply not including the correct class of the datapoint into the approximation. To account for this we explicitly sum over a subset of classes and sample the remaining. We show that this simple approach is competitive with recently introduced non likelihood-based approximations.
APA
Botev, A., Zheng, B. & Barber, D.. (2017). Complementary Sum Sampling for Likelihood Approximation in Large Scale Classification. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 54:1030-1038 Available from https://proceedings.mlr.press/v54/botev17a.html.

Related Material