A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization

Hanqin Cai, Yuchen Lou, Daniel Mckenzie, Wotao Yin
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:1193-1203, 2021.

Abstract

We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over one million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9% with significantly less distortion.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-cai21d, title = {A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization}, author = {Cai, Hanqin and Lou, Yuchen and Mckenzie, Daniel and Yin, Wotao}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {1193--1203}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/cai21d/cai21d.pdf}, url = {https://proceedings.mlr.press/v139/cai21d.html}, abstract = {We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over one million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9% with significantly less distortion.} }
Endnote
%0 Conference Paper %T A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization %A Hanqin Cai %A Yuchen Lou %A Daniel Mckenzie %A Wotao Yin %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-cai21d %I PMLR %P 1193--1203 %U https://proceedings.mlr.press/v139/cai21d.html %V 139 %X We consider the zeroth-order optimization problem in the huge-scale setting, where the dimension of the problem is so large that performing even basic vector operations on the decision variables is infeasible. In this paper, we propose a novel algorithm, coined ZO-BCD, that exhibits favorable overall query complexity and has a much smaller per-iteration computational complexity. In addition, we discuss how the memory footprint of ZO-BCD can be reduced even further by the clever use of circulant measurement matrices. As an application of our new method, we propose the idea of crafting adversarial attacks on neural network based classifiers in a wavelet domain, which can result in problem dimensions of over one million. In particular, we show that crafting adversarial examples to audio classifiers in a wavelet domain can achieve the state-of-the-art attack success rate of 97.9% with significantly less distortion.
APA
Cai, H., Lou, Y., Mckenzie, D. & Yin, W.. (2021). A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:1193-1203 Available from https://proceedings.mlr.press/v139/cai21d.html.

Related Material