Optimal and Adaptive Algorithms for Online Boosting


Alina Beygelzimer, Satyen Kale, Haipeng Luo ;
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:2323-2331, 2015.


We study online boosting, the task of converting any weak online learner into a strong online learner. Based on a novel and natural definition of weak online learnability, we develop two online boosting algorithms. The first algorithm is an online version of boost-by-majority. By proving a matching lower bound, we show that this algorithm is essentially optimal in terms of the number of weak learners and the sample complexity needed to achieve a specified accuracy. The second algorithm is adaptive and parameter-free, albeit not optimal.

Related Material