Dissecting the Winning Solution of the HiggsML Challenge
Proceedings of the NIPS 2014 Workshop on High-energy Physics and Machine Learning, PMLR 42:57-67, 2015.
The recent Higgs Machine Learning Challenge pitted one of the largest crowds seen in machine learning contests against one another. In this paper, we present the winning solution and investigate the effect of extra features, the choice of neural network activation function, regularization and data set size. We demonstrate improved classification accuracy using a very similar network architecture on the permutation invariant MNIST benchmark. Furthermore, we advocate the use of a simple method that lies on the boundary between bagging and cross-validation to both estimate the generalization error and improve accuracy.