Robust descent using smoothed multiplicative noise
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:703-711, 2019.
In this work, we propose a novel robust gradient descent procedure which makes use of a smoothed multiplicative noise applied directly to observations before constructing a sum of soft-truncated gradient coordinates. We show that the procedure has competitive theoretical guarantees, with the major advantage of a simple implementation that does not require an iterative sub-routine for robustification. Empirical tests reinforce the theory, showing more efficient generalization over a much wider class of data distributions.