Advances in PAC-Bayesian certification of deep neural networks: tighter closed-form inequalities and optimization of bounds on non-differentiable losses

Diego García-Pérez, Emilio Parrado-Hernandez, John Shawe-Taylor
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:1650-1662, 2026.

Abstract

This paper presents three theoretical contributions that improve the usability of risk certificates for neural networks based on {PAC-Bayes} bounds. First, two bounds on the KL divergence between {Bernoulli} distributions enable the derivation of the tightest explicit bounds on the true risk of classifiers across different ranges of empirical risk. Then, a novel method to optimize bounds on non-differentiable objectives is introduced, allowing to optimize risk bounds directly on the 0-1 loss instead of resourcing to a surrogate differentiable loss, such as the bounded cross-entropy. These theoretical contributions are illustrated with an empirical evaluation on the {MNIST} and {CIFAR-10} datasets. In fact, this paper presents the first non data-dependent generalization bounds on the 0-1 loss for neural networks fitted on {CIFAR-10}.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-garcia-perez26a, title = {Advances in {PAC}-{Bayesian} certification of deep neural networks: tighter closed-form inequalities and optimization of bounds on non-differentiable losses}, author = {Garc\'{i}a-P\'{e}rez, Diego and Parrado-Hernandez, Emilio and Shawe-{Taylor}, John}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {1650--1662}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/garcia-perez26a/garcia-perez26a.pdf}, url = {https://proceedings.mlr.press/v337/garcia-perez26a.html}, abstract = {This paper presents three theoretical contributions that improve the usability of risk certificates for neural networks based on {PAC-Bayes} bounds. First, two bounds on the KL divergence between {Bernoulli} distributions enable the derivation of the tightest explicit bounds on the true risk of classifiers across different ranges of empirical risk. Then, a novel method to optimize bounds on non-differentiable objectives is introduced, allowing to optimize risk bounds directly on the 0-1 loss instead of resourcing to a surrogate differentiable loss, such as the bounded cross-entropy. These theoretical contributions are illustrated with an empirical evaluation on the {MNIST} and {CIFAR-10} datasets. In fact, this paper presents the first non data-dependent generalization bounds on the 0-1 loss for neural networks fitted on {CIFAR-10}.} }
Endnote
%0 Conference Paper %T Advances in PAC-Bayesian certification of deep neural networks: tighter closed-form inequalities and optimization of bounds on non-differentiable losses %A Diego García-Pérez %A Emilio Parrado-Hernandez %A John Shawe-Taylor %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-garcia-perez26a %I PMLR %P 1650--1662 %U https://proceedings.mlr.press/v337/garcia-perez26a.html %V 337 %X This paper presents three theoretical contributions that improve the usability of risk certificates for neural networks based on {PAC-Bayes} bounds. First, two bounds on the KL divergence between {Bernoulli} distributions enable the derivation of the tightest explicit bounds on the true risk of classifiers across different ranges of empirical risk. Then, a novel method to optimize bounds on non-differentiable objectives is introduced, allowing to optimize risk bounds directly on the 0-1 loss instead of resourcing to a surrogate differentiable loss, such as the bounded cross-entropy. These theoretical contributions are illustrated with an empirical evaluation on the {MNIST} and {CIFAR-10} datasets. In fact, this paper presents the first non data-dependent generalization bounds on the 0-1 loss for neural networks fitted on {CIFAR-10}.
APA
García-Pérez, D., Parrado-Hernandez, E. & Shawe-Taylor, J.. (2026). Advances in PAC-Bayesian certification of deep neural networks: tighter closed-form inequalities and optimization of bounds on non-differentiable losses. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:1650-1662 Available from https://proceedings.mlr.press/v337/garcia-perez26a.html.

Related Material