Distributional Adversarial Loss

Saba Ahmadi, Siddharth Bhandari, Avrim Blum, Chen Dan, Prabhav Jain
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:2278-2286, 2025.

Abstract

We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show sample complexity bounds in the PAC-learning setting for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work on robust learning that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods. Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier’s robustness against adversarial attacks and show its effectiveness empirically.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-ahmadi25a, title = {Distributional Adversarial Loss}, author = {Ahmadi, Saba and Bhandari, Siddharth and Blum, Avrim and Dan, Chen and Jain, Prabhav}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {2278--2286}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/ahmadi25a/ahmadi25a.pdf}, url = {https://proceedings.mlr.press/v258/ahmadi25a.html}, abstract = {We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show sample complexity bounds in the PAC-learning setting for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work on robust learning that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods. Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier’s robustness against adversarial attacks and show its effectiveness empirically.} }
Endnote
%0 Conference Paper %T Distributional Adversarial Loss %A Saba Ahmadi %A Siddharth Bhandari %A Avrim Blum %A Chen Dan %A Prabhav Jain %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-ahmadi25a %I PMLR %P 2278--2286 %U https://proceedings.mlr.press/v258/ahmadi25a.html %V 258 %X We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show sample complexity bounds in the PAC-learning setting for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work on robust learning that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods. Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier’s robustness against adversarial attacks and show its effectiveness empirically.
APA
Ahmadi, S., Bhandari, S., Blum, A., Dan, C. & Jain, P.. (2025). Distributional Adversarial Loss. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:2278-2286 Available from https://proceedings.mlr.press/v258/ahmadi25a.html.

Related Material