Risk-limiting financial audits via weighted sampling without replacement

Shubhanshu Shekhar, Ziyu Xu, Zachary Lipton, Pierre Liang, Aaditya Ramdas
Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, PMLR 216:1932-1941, 2023.

Abstract

We introduce the notion of risk-limiting financial audits (RLFA): procedures that manually evaluate a subset of $N$ financial transactions to check the validity of a claimed assertion $\mathcal{A}$ about the transactions. More specifically, RLFA satisfy two properties: (i) if $\mathcal{A}$ is false, they correctly disprove it with probability at least $1-\delta$, and (ii) they validate the correctness of $\mathcal{A}$ with probability $1$, if it is true. We propose a general RLFA strategy, by constructing new confidence sequences (CSs) for the weighted average of $N$ unknown values, based on samples drawn without replacement from a (randomized) weighted sampling scheme. Next, we develop methods to improve the quality of CSs by incorporating side information about the unknown values. We show that when the side information is sufficiently accurate, it can directly drive the sampling. For the case where the accuracy is unknown a priori, we introduce an alternative approach using control variates. Crucially, our construction adapts to the quality of side information by strongly leveraging the side information if it is highly predictive, and learning to ignore it if it is uninformative. Our methods also recover the state-of-the-art bounds for the special case of uniformly sampled observations with no side information, which has already found applications in election auditing. The harder weighted case with general side information solves the more challenging problem of AI-assisted financial auditing.

Cite this Paper

BibTeX
@InProceedings{pmlr-v216-shekhar23a, title = {Risk-limiting financial audits via weighted sampling without replacement}, author = {Shekhar, Shubhanshu and Xu, Ziyu and Lipton, Zachary and Liang, Pierre and Ramdas, Aaditya}, booktitle = {Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence}, pages = {1932--1941}, year = {2023}, editor = {Evans, Robin J. and Shpitser, Ilya}, volume = {216}, series = {Proceedings of Machine Learning Research}, month = {31 Jul--04 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v216/shekhar23a/shekhar23a.pdf}, url = {https://proceedings.mlr.press/v216/shekhar23a.html}, abstract = {We introduce the notion of risk-limiting financial audits (RLFA): procedures that manually evaluate a subset of $N$ financial transactions to check the validity of a claimed assertion $\mathcal{A}$ about the transactions. More specifically, RLFA satisfy two properties: (i) if $\mathcal{A}$ is false, they correctly disprove it with probability at least $1-\delta$, and (ii) they validate the correctness of $\mathcal{A}$ with probability $1$, if it is true. We propose a general RLFA strategy, by constructing new confidence sequences (CSs) for the weighted average of $N$ unknown values, based on samples drawn without replacement from a (randomized) weighted sampling scheme. Next, we develop methods to improve the quality of CSs by incorporating side information about the unknown values. We show that when the side information is sufficiently accurate, it can directly drive the sampling. For the case where the accuracy is unknown a priori, we introduce an alternative approach using control variates. Crucially, our construction adapts to the quality of side information by strongly leveraging the side information if it is highly predictive, and learning to ignore it if it is uninformative. Our methods also recover the state-of-the-art bounds for the special case of uniformly sampled observations with no side information, which has already found applications in election auditing. The harder weighted case with general side information solves the more challenging problem of AI-assisted financial auditing.} }
Endnote
%0 Conference Paper %T Risk-limiting financial audits via weighted sampling without replacement %A Shubhanshu Shekhar %A Ziyu Xu %A Zachary Lipton %A Pierre Liang %A Aaditya Ramdas %B Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2023 %E Robin J. Evans %E Ilya Shpitser %F pmlr-v216-shekhar23a %I PMLR %P 1932--1941 %U https://proceedings.mlr.press/v216/shekhar23a.html %V 216 %X We introduce the notion of risk-limiting financial audits (RLFA): procedures that manually evaluate a subset of $N$ financial transactions to check the validity of a claimed assertion $\mathcal{A}$ about the transactions. More specifically, RLFA satisfy two properties: (i) if $\mathcal{A}$ is false, they correctly disprove it with probability at least $1-\delta$, and (ii) they validate the correctness of $\mathcal{A}$ with probability $1$, if it is true. We propose a general RLFA strategy, by constructing new confidence sequences (CSs) for the weighted average of $N$ unknown values, based on samples drawn without replacement from a (randomized) weighted sampling scheme. Next, we develop methods to improve the quality of CSs by incorporating side information about the unknown values. We show that when the side information is sufficiently accurate, it can directly drive the sampling. For the case where the accuracy is unknown a priori, we introduce an alternative approach using control variates. Crucially, our construction adapts to the quality of side information by strongly leveraging the side information if it is highly predictive, and learning to ignore it if it is uninformative. Our methods also recover the state-of-the-art bounds for the special case of uniformly sampled observations with no side information, which has already found applications in election auditing. The harder weighted case with general side information solves the more challenging problem of AI-assisted financial auditing.
APA
Shekhar, S., Xu, Z., Lipton, Z., Liang, P. & Ramdas, A.. (2023). Risk-limiting financial audits via weighted sampling without replacement. Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 216:1932-1941 Available from https://proceedings.mlr.press/v216/shekhar23a.html.

Related Material