MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation

Wenhao Wang, Peizhi Niu, Gongyi Zou, Xiyuan Yang, Jingxing Wang, Haoting Shi, Yaxin Du, Jingyi Chai, Xianghe Pang, Shuo Tang, Yanfeng Wang, Siheng Chen
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:129301-129320, 2026.

Abstract

The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social applications, where tools interact with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools. MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark’s crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-wang26gb, title = {{MCP}-Persona: Benchmarking {LLM} Agents on Real-World Personal Applications via Environment Simulation}, author = {Wang, Wenhao and Niu, Peizhi and Zou, Gongyi and Yang, Xiyuan and Wang, Jingxing and Shi, Haoting and Du, Yaxin and Chai, Jingyi and Pang, Xianghe and Tang, Shuo and Wang, Yanfeng and Chen, Siheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {129301--129320}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wang26gb/wang26gb.pdf}, url = {https://proceedings.mlr.press/v306/wang26gb.html}, abstract = {The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social applications, where tools interact with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools. MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark’s crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona} }
Endnote
%0 Conference Paper %T MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation %A Wenhao Wang %A Peizhi Niu %A Gongyi Zou %A Xiyuan Yang %A Jingxing Wang %A Haoting Shi %A Yaxin Du %A Jingyi Chai %A Xianghe Pang %A Shuo Tang %A Yanfeng Wang %A Siheng Chen %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wang26gb %I PMLR %P 129301--129320 %U https://proceedings.mlr.press/v306/wang26gb.html %V 306 %X The Model Context Protocol (MCP) has emerged as a transformative standard for connecting large language models (LLMs) with external data sources and tools, and has been rapidly adopted across personal applications and development platforms. However, existing benchmarks predominantly focus on generic information-seeking tools and fail to capture the practical challenges posed by personal social applications, where tools interact with individual accounts or local databases. To bridge this critical gap, we introduce MCP-Persona, the first benchmark specifically designed for evaluating agent performance on real-world, personalized MCP tools. MCP-Persona encompasses a diverse set of widely-used applications, ranging from social media platforms like Reddit and Xiaohongshu (Rednote) to enterprise collaboration suites such as Lark (Feishu) and Slack. Our extensive experiments on various state-of-the-art (SOTA) agents demonstrate their significant struggles with personalized tool use, thereby highlighting the benchmark’s crucial role in identifying and addressing these limitations. MCP-Persona is publicly available at https://github.com/wwh0411/MCP-Persona
APA
Wang, W., Niu, P., Zou, G., Yang, X., Wang, J., Shi, H., Du, Y., Chai, J., Pang, X., Tang, S., Wang, Y. & Chen, S.. (2026). MCP-Persona: Benchmarking LLM Agents on Real-World Personal Applications via Environment Simulation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:129301-129320 Available from https://proceedings.mlr.press/v306/wang26gb.html.

Related Material