Learning Mixtures of Linear Classifiers


Yuekai Sun, Stratis Ioannidis, Andrea Montanari ;
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):721-729, 2014.


We consider a discriminative learning (regression) problem, whereby the regression function is a convex combination of k linear classifiers. Existing approaches are based on the EM algorithm, or similar techniques, without provable guarantees. We develop a simple method based on spectral techniques and a ‘mirroring’ trick, that discovers the subspace spanned by the classifiers’ parameter vectors. Under a probabilistic assumption on the feature vector distribution, we prove that this approach has nearly optimal statistical efficiency.

Related Material