Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

Blaise Delattre, Hengyu Wu, Paul Caillon, Wei Yang Bryan Lim, Yang Cao
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:23613-23636, 2026.

Abstract

Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete–continuous inputs based on an analytically tractable Neyman–Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman–Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text–image safety filtering.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-delattre26a, title = {Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing}, author = {Delattre, Blaise and Wu, Hengyu and Caillon, Paul and Lim, Wei Yang Bryan and Cao, Yang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {23613--23636}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/delattre26a/delattre26a.pdf}, url = {https://proceedings.mlr.press/v306/delattre26a.html}, abstract = {Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete–continuous inputs based on an analytically tractable Neyman–Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman–Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text–image safety filtering.} }
Endnote
%0 Conference Paper %T Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing %A Blaise Delattre %A Hengyu Wu %A Paul Caillon %A Wei Yang Bryan Lim %A Yang Cao %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-delattre26a %I PMLR %P 23613--23636 %U https://proceedings.mlr.press/v306/delattre26a.html %V 306 %X Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete–continuous inputs based on an analytically tractable Neyman–Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman–Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text–image safety filtering.
APA
Delattre, B., Wu, H., Caillon, P., Lim, W.Y.B. & Cao, Y.. (2026). Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:23613-23636 Available from https://proceedings.mlr.press/v306/delattre26a.html.

Related Material