Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM

Tianyu Wang, Luhao Zhang, Rachel Cummings
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:1018-1026, 2026.

Abstract

Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, \textbf{CorrDP}, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the \textbf{CorrDP} framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that \textbf{CorrDP}-based DP-ERM algorithms consistently outperform the standard DP framework.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-wang26c, title = { Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM }, author = {Wang, Tianyu and Zhang, Luhao and Cummings, Rachel}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {1018--1026}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/wang26c/wang26c.pdf}, url = {https://proceedings.mlr.press/v300/wang26c.html}, abstract = { Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, \textbf{CorrDP}, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the \textbf{CorrDP} framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that \textbf{CorrDP}-based DP-ERM algorithms consistently outperform the standard DP framework. } }
Endnote
%0 Conference Paper %T Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM %A Tianyu Wang %A Luhao Zhang %A Rachel Cummings %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-wang26c %I PMLR %P 1018--1026 %U https://proceedings.mlr.press/v300/wang26c.html %V 300 %X Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, \textbf{CorrDP}, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the \textbf{CorrDP} framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that \textbf{CorrDP}-based DP-ERM algorithms consistently outperform the standard DP framework.
APA
Wang, T., Zhang, L. & Cummings, R.. (2026). Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:1018-1026 Available from https://proceedings.mlr.press/v300/wang26c.html.

Related Material