Certified Multi-Source Integrity for Structured Agent Actions

Anmol Pandey, Aditya Jain, Liang Chen, Carsten Maple, Christo Panchev
Proceedings of the Fourth UK AI Conference 2026, PMLR 348:134-147, 2026.

Abstract

LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from action-critical fields in documents and tool outputs that an adversary can corrupt, and indirect prompt injection can drive the model itself to extract attacker-chosen values. Current defenses gate on a source’s trust label or certify free-text answer quality. None certifies the integrity of a coupled, policy-bound structured action under a corruption budget that accounts for shared upstream sources. We characterize when such an action is safely certifiable and give the maximally live safe certifier. It admits an action only when each field clears the rule its evidence structure supports: a bounded corruption radius over corruption-distinct evidence classes, counted by a minimum hitting set so that re-publishing or laundered copies cannot manufacture quorum, deterministic reconciliation for complementary fields, and a trusted anchor where the evidence leaves a field single-sourced. We formalize two robustness notions, validate each mechanism by ablation, and measure how often the multi-source precondition holds on sanctions designations (70,966 entities) and software supply-chain provenance (450 packages). Under upper-bound proxies, genuine corroboration is a minority phenomenon in both, and naive attestation counting overstates it, since witnesses that look independent collapse to two corruption-distinct domains once shared origin is counted. Across five current models in a real agent loop, a realistic injection fools every model but one and a naive agent then executes the fraudulent action on most attacks. The certifier admits no unsafe action and recovers the correct value where corroboration permits, while action-gating and provenance baselines are broken in every world of our harness.

Cite this Paper


BibTeX
@InProceedings{pmlr-v348-pandey26a, title = {Certified Multi-Source Integrity for Structured Agent Actions}, author = {Pandey, Anmol and Jain, Aditya and Chen, Liang and Maple, Carsten and Panchev, Christo}, booktitle = {Proceedings of the Fourth UK AI Conference 2026}, pages = {134--147}, year = {2026}, editor = {Benford, Alistair and Büyükateş, Baturalp and Cabrera, Christian and Kiden, Sarah and Salili-James, Arianna and Zakka, Vincent and Zhou, Feng}, volume = {348}, series = {Proceedings of Machine Learning Research}, month = {29--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v348/main/assets/pandey26a/pandey26a.pdf}, url = {https://proceedings.mlr.press/v348/pandey26a.html}, abstract = {LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from action-critical fields in documents and tool outputs that an adversary can corrupt, and indirect prompt injection can drive the model itself to extract attacker-chosen values. Current defenses gate on a source’s trust label or certify free-text answer quality. None certifies the integrity of a coupled, policy-bound structured action under a corruption budget that accounts for shared upstream sources. We characterize when such an action is safely certifiable and give the maximally live safe certifier. It admits an action only when each field clears the rule its evidence structure supports: a bounded corruption radius over corruption-distinct evidence classes, counted by a minimum hitting set so that re-publishing or laundered copies cannot manufacture quorum, deterministic reconciliation for complementary fields, and a trusted anchor where the evidence leaves a field single-sourced. We formalize two robustness notions, validate each mechanism by ablation, and measure how often the multi-source precondition holds on sanctions designations (70,966 entities) and software supply-chain provenance (450 packages). Under upper-bound proxies, genuine corroboration is a minority phenomenon in both, and naive attestation counting overstates it, since witnesses that look independent collapse to two corruption-distinct domains once shared origin is counted. Across five current models in a real agent loop, a realistic injection fools every model but one and a naive agent then executes the fraudulent action on most attacks. The certifier admits no unsafe action and recovers the correct value where corroboration permits, while action-gating and provenance baselines are broken in every world of our harness.} }
Endnote
%0 Conference Paper %T Certified Multi-Source Integrity for Structured Agent Actions %A Anmol Pandey %A Aditya Jain %A Liang Chen %A Carsten Maple %A Christo Panchev %B Proceedings of the Fourth UK AI Conference 2026 %C Proceedings of Machine Learning Research %D 2026 %E Alistair Benford %E Baturalp Büyükateş %E Christian Cabrera %E Sarah Kiden %E Arianna Salili-James %E Vincent Zakka %E Feng Zhou %F pmlr-v348-pandey26a %I PMLR %P 134--147 %U https://proceedings.mlr.press/v348/pandey26a.html %V 348 %X LLM agents increasingly take privileged, often irreversible structured actions, such as paying an invoice. They assemble each action from action-critical fields in documents and tool outputs that an adversary can corrupt, and indirect prompt injection can drive the model itself to extract attacker-chosen values. Current defenses gate on a source’s trust label or certify free-text answer quality. None certifies the integrity of a coupled, policy-bound structured action under a corruption budget that accounts for shared upstream sources. We characterize when such an action is safely certifiable and give the maximally live safe certifier. It admits an action only when each field clears the rule its evidence structure supports: a bounded corruption radius over corruption-distinct evidence classes, counted by a minimum hitting set so that re-publishing or laundered copies cannot manufacture quorum, deterministic reconciliation for complementary fields, and a trusted anchor where the evidence leaves a field single-sourced. We formalize two robustness notions, validate each mechanism by ablation, and measure how often the multi-source precondition holds on sanctions designations (70,966 entities) and software supply-chain provenance (450 packages). Under upper-bound proxies, genuine corroboration is a minority phenomenon in both, and naive attestation counting overstates it, since witnesses that look independent collapse to two corruption-distinct domains once shared origin is counted. Across five current models in a real agent loop, a realistic injection fools every model but one and a naive agent then executes the fraudulent action on most attacks. The certifier admits no unsafe action and recovers the correct value where corroboration permits, while action-gating and provenance baselines are broken in every world of our harness.
APA
Pandey, A., Jain, A., Chen, L., Maple, C. & Panchev, C.. (2026). Certified Multi-Source Integrity for Structured Agent Actions. Proceedings of the Fourth UK AI Conference 2026, in Proceedings of Machine Learning Research 348:134-147 Available from https://proceedings.mlr.press/v348/pandey26a.html.

Related Material