Robust descent using smoothed multiplicative noise

[edit]

Matthew Holland ;
Proceedings of Machine Learning Research, PMLR 89:703-711, 2019.

Abstract

In this work, we propose a novel robust gradient descent procedure which makes use of a smoothed multiplicative noise applied directly to observations before constructing a sum of soft-truncated gradient coordinates. We show that the procedure has competitive theoretical guarantees, with the major advantage of a simple implementation that does not require an iterative sub-routine for robustification. Empirical tests reinforce the theory, showing more efficient generalization over a much wider class of data distributions.

Related Material