OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation

Yichen Wu, Qianqian Gao, Xudong Pan, Geng Hong, Min Yang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:135874-135897, 2026.

Abstract

As large language models (LLMs) are increasingly deployed as interactive agents, open-ended human-AI interactions can involve deceptive behaviors with serious real-world consequences, yet existing evaluations remain largely scenario-specific and model-centric. We introduce OpenDeception, a lightweight framework for jointly evaluating deception risk from both sides of human-AI dialogue. It consists of a scenario benchmark with 50 real-world deception cases, an IntentNet that infers deceptive intent from agent reasoning, and a TrustNet that estimates user susceptibility. To address data scarcity, we synthesize high-risk dialogues via LLM-based role-and-goal simulation, and train the TrustNet using contrastive learning on controlled response pairs, avoiding unreliable scalar labels. Experiments on 11 LLMs and three large reasoning models show that over 90% of goal-driven interactions in most models exhibit deceptive intent, with stronger models displaying higher risk. A real-world case study adapted from a documented AI-induced suicide incident further demonstrates that our joint evaluation can proactively trigger warnings before critical trust thresholds are reached.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-wu26p, title = {{O}pen{D}eception: Learning Deception and Trust in Human–{AI} Interaction via Multi-Agent Simulation}, author = {Wu, Yichen and Gao, Qianqian and Pan, Xudong and Hong, Geng and Yang, Min}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {135874--135897}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wu26p/wu26p.pdf}, url = {https://proceedings.mlr.press/v306/wu26p.html}, abstract = {As large language models (LLMs) are increasingly deployed as interactive agents, open-ended human-AI interactions can involve deceptive behaviors with serious real-world consequences, yet existing evaluations remain largely scenario-specific and model-centric. We introduce OpenDeception, a lightweight framework for jointly evaluating deception risk from both sides of human-AI dialogue. It consists of a scenario benchmark with 50 real-world deception cases, an IntentNet that infers deceptive intent from agent reasoning, and a TrustNet that estimates user susceptibility. To address data scarcity, we synthesize high-risk dialogues via LLM-based role-and-goal simulation, and train the TrustNet using contrastive learning on controlled response pairs, avoiding unreliable scalar labels. Experiments on 11 LLMs and three large reasoning models show that over 90% of goal-driven interactions in most models exhibit deceptive intent, with stronger models displaying higher risk. A real-world case study adapted from a documented AI-induced suicide incident further demonstrates that our joint evaluation can proactively trigger warnings before critical trust thresholds are reached.} }
Endnote
%0 Conference Paper %T OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation %A Yichen Wu %A Qianqian Gao %A Xudong Pan %A Geng Hong %A Min Yang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wu26p %I PMLR %P 135874--135897 %U https://proceedings.mlr.press/v306/wu26p.html %V 306 %X As large language models (LLMs) are increasingly deployed as interactive agents, open-ended human-AI interactions can involve deceptive behaviors with serious real-world consequences, yet existing evaluations remain largely scenario-specific and model-centric. We introduce OpenDeception, a lightweight framework for jointly evaluating deception risk from both sides of human-AI dialogue. It consists of a scenario benchmark with 50 real-world deception cases, an IntentNet that infers deceptive intent from agent reasoning, and a TrustNet that estimates user susceptibility. To address data scarcity, we synthesize high-risk dialogues via LLM-based role-and-goal simulation, and train the TrustNet using contrastive learning on controlled response pairs, avoiding unreliable scalar labels. Experiments on 11 LLMs and three large reasoning models show that over 90% of goal-driven interactions in most models exhibit deceptive intent, with stronger models displaying higher risk. A real-world case study adapted from a documented AI-induced suicide incident further demonstrates that our joint evaluation can proactively trigger warnings before critical trust thresholds are reached.
APA
Wu, Y., Gao, Q., Pan, X., Hong, G. & Yang, M.. (2026). OpenDeception: Learning Deception and Trust in Human–AI Interaction via Multi-Agent Simulation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:135874-135897 Available from https://proceedings.mlr.press/v306/wu26p.html.

Related Material