MemPot: Defending Against Memory Extraction Attack with Optimized Honeypots

Yuhao Wang, Shengfang Zhai, Guanghao Jin, Yinpeng Dong, Linyi Yang, Jiaheng Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:132129-132152, 2026.

Abstract

Large Language Model (LLM)-based agents employ external and internal memory systems to handle complex, goal-oriented tasks, yet this exposes them to severe extraction attacks, and corresponding defenses are currently lacking. In this paper, we propose MemPot, the first theoretically verified defense framework against memory extraction attacks by injecting optimized honeypots into the memory. Through a two-stage optimization process, MemPot generates trap documents that maximize the retrieval probability for attackers while remaining inconspicuous to benign users. We model the detection process as Wald’s Sequential Probability Ratio Test (SPRT) and theoretically prove that MemPot achieves a lower average number of sampling rounds compared to optimal static detectors. Empirically, MemPot significantly outperforms state-of-the-art baselines, achieving a 50% improvement in detection AUROC and an 80% increase in True Positive Rate under low False Positive Rate constraints. Furthermore, our experiments confirm that MemPot incurs zero online inference latency and preserves the agent’s utility on standard tasks, verifying its superiority in safety, harmlessness and efficiency.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-wang26kg, title = {{M}em{P}ot: Defending Against Memory Extraction Attack with Optimized Honeypots}, author = {Wang, Yuhao and Zhai, Shengfang and Jin, Guanghao and Dong, Yinpeng and Yang, Linyi and Zhang, Jiaheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {132129--132152}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wang26kg/wang26kg.pdf}, url = {https://proceedings.mlr.press/v306/wang26kg.html}, abstract = {Large Language Model (LLM)-based agents employ external and internal memory systems to handle complex, goal-oriented tasks, yet this exposes them to severe extraction attacks, and corresponding defenses are currently lacking. In this paper, we propose MemPot, the first theoretically verified defense framework against memory extraction attacks by injecting optimized honeypots into the memory. Through a two-stage optimization process, MemPot generates trap documents that maximize the retrieval probability for attackers while remaining inconspicuous to benign users. We model the detection process as Wald’s Sequential Probability Ratio Test (SPRT) and theoretically prove that MemPot achieves a lower average number of sampling rounds compared to optimal static detectors. Empirically, MemPot significantly outperforms state-of-the-art baselines, achieving a 50% improvement in detection AUROC and an 80% increase in True Positive Rate under low False Positive Rate constraints. Furthermore, our experiments confirm that MemPot incurs zero online inference latency and preserves the agent’s utility on standard tasks, verifying its superiority in safety, harmlessness and efficiency.} }
Endnote
%0 Conference Paper %T MemPot: Defending Against Memory Extraction Attack with Optimized Honeypots %A Yuhao Wang %A Shengfang Zhai %A Guanghao Jin %A Yinpeng Dong %A Linyi Yang %A Jiaheng Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wang26kg %I PMLR %P 132129--132152 %U https://proceedings.mlr.press/v306/wang26kg.html %V 306 %X Large Language Model (LLM)-based agents employ external and internal memory systems to handle complex, goal-oriented tasks, yet this exposes them to severe extraction attacks, and corresponding defenses are currently lacking. In this paper, we propose MemPot, the first theoretically verified defense framework against memory extraction attacks by injecting optimized honeypots into the memory. Through a two-stage optimization process, MemPot generates trap documents that maximize the retrieval probability for attackers while remaining inconspicuous to benign users. We model the detection process as Wald’s Sequential Probability Ratio Test (SPRT) and theoretically prove that MemPot achieves a lower average number of sampling rounds compared to optimal static detectors. Empirically, MemPot significantly outperforms state-of-the-art baselines, achieving a 50% improvement in detection AUROC and an 80% increase in True Positive Rate under low False Positive Rate constraints. Furthermore, our experiments confirm that MemPot incurs zero online inference latency and preserves the agent’s utility on standard tasks, verifying its superiority in safety, harmlessness and efficiency.
APA
Wang, Y., Zhai, S., Jin, G., Dong, Y., Yang, L. & Zhang, J.. (2026). MemPot: Defending Against Memory Extraction Attack with Optimized Honeypots. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:132129-132152 Available from https://proceedings.mlr.press/v306/wang26kg.html.

Related Material