How Far Can LLM Agents Reason with Tables? Benchmarking Multi-Turn Agentic Table Question Answering in the Wild

Jingwang Huang, Jie Zhang, Haoyang Zeng, Changzai Pan, Xianjie Wu, Guanting Dong, Jiaheng Liu, Wei Zhang, Mingyu Zheng, Chunxiao Liu, Kaiwen Wei, Jiang Zhong, Jian Yang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:48385-48418, 2026.

Abstract

Recent advances in large language models (LLMs) have substantially expanded the scope of Table Question Answering (TableQA). However, existing benchmarks primarily treat TableQA as a passive, single-turn natural language understanding task, lacking the capacity to evaluate autonomous reasoning and tool-call trajectories in realistic, multi-turn scenarios. To bridge this gap, we introduce TableAgent-Bench, a large-scale bilingual benchmark that reformulates TableQA as proactive, agentic interactions over structurally complex, multi-table environments. With a topology-aware construction strategy, TableAgent-Bench captures dynamic intent evolution through 1,310 multi-turn dialogues grounded in 2,275 industrial tables. Furthermore, we propose the Table-centric Agent Evaluation Framework (TAEF) to assess agent interactions with complex table structures. Specifically, TAEF integrates a specialized agent toolset and 4 metric categories to systematically diagnose intermediate failure modes, assessing performance across table localization, tool-invocation rationality, and trajectory-level pass rate. Extensive experiments with 25 state-of-the-art LLM agents reveal a substantial capability gap, with even the strongest model Gemini-3-Pro-Preview achieving only 53.4% information coverage. We expect TableAgent-Bench to serve as a rigorous testbed for developing and evaluating agents capable of robust table-centric reasoning.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-huang26cf, title = {How Far Can {LLM} Agents Reason with Tables? {B}enchmarking Multi-Turn Agentic Table Question Answering in the Wild}, author = {Huang, Jingwang and Zhang, Jie and Zeng, Haoyang and Pan, Changzai and Wu, Xianjie and Dong, Guanting and Liu, Jiaheng and Zhang, Wei and Zheng, Mingyu and Liu, Chunxiao and Wei, Kaiwen and Zhong, Jiang and Yang, Jian}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {48385--48418}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/huang26cf/huang26cf.pdf}, url = {https://proceedings.mlr.press/v306/huang26cf.html}, abstract = {Recent advances in large language models (LLMs) have substantially expanded the scope of Table Question Answering (TableQA). However, existing benchmarks primarily treat TableQA as a passive, single-turn natural language understanding task, lacking the capacity to evaluate autonomous reasoning and tool-call trajectories in realistic, multi-turn scenarios. To bridge this gap, we introduce TableAgent-Bench, a large-scale bilingual benchmark that reformulates TableQA as proactive, agentic interactions over structurally complex, multi-table environments. With a topology-aware construction strategy, TableAgent-Bench captures dynamic intent evolution through 1,310 multi-turn dialogues grounded in 2,275 industrial tables. Furthermore, we propose the Table-centric Agent Evaluation Framework (TAEF) to assess agent interactions with complex table structures. Specifically, TAEF integrates a specialized agent toolset and 4 metric categories to systematically diagnose intermediate failure modes, assessing performance across table localization, tool-invocation rationality, and trajectory-level pass rate. Extensive experiments with 25 state-of-the-art LLM agents reveal a substantial capability gap, with even the strongest model Gemini-3-Pro-Preview achieving only 53.4% information coverage. We expect TableAgent-Bench to serve as a rigorous testbed for developing and evaluating agents capable of robust table-centric reasoning.} }
Endnote
%0 Conference Paper %T How Far Can LLM Agents Reason with Tables? Benchmarking Multi-Turn Agentic Table Question Answering in the Wild %A Jingwang Huang %A Jie Zhang %A Haoyang Zeng %A Changzai Pan %A Xianjie Wu %A Guanting Dong %A Jiaheng Liu %A Wei Zhang %A Mingyu Zheng %A Chunxiao Liu %A Kaiwen Wei %A Jiang Zhong %A Jian Yang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-huang26cf %I PMLR %P 48385--48418 %U https://proceedings.mlr.press/v306/huang26cf.html %V 306 %X Recent advances in large language models (LLMs) have substantially expanded the scope of Table Question Answering (TableQA). However, existing benchmarks primarily treat TableQA as a passive, single-turn natural language understanding task, lacking the capacity to evaluate autonomous reasoning and tool-call trajectories in realistic, multi-turn scenarios. To bridge this gap, we introduce TableAgent-Bench, a large-scale bilingual benchmark that reformulates TableQA as proactive, agentic interactions over structurally complex, multi-table environments. With a topology-aware construction strategy, TableAgent-Bench captures dynamic intent evolution through 1,310 multi-turn dialogues grounded in 2,275 industrial tables. Furthermore, we propose the Table-centric Agent Evaluation Framework (TAEF) to assess agent interactions with complex table structures. Specifically, TAEF integrates a specialized agent toolset and 4 metric categories to systematically diagnose intermediate failure modes, assessing performance across table localization, tool-invocation rationality, and trajectory-level pass rate. Extensive experiments with 25 state-of-the-art LLM agents reveal a substantial capability gap, with even the strongest model Gemini-3-Pro-Preview achieving only 53.4% information coverage. We expect TableAgent-Bench to serve as a rigorous testbed for developing and evaluating agents capable of robust table-centric reasoning.
APA
Huang, J., Zhang, J., Zeng, H., Pan, C., Wu, X., Dong, G., Liu, J., Zhang, W., Zheng, M., Liu, C., Wei, K., Zhong, J. & Yang, J.. (2026). How Far Can LLM Agents Reason with Tables? Benchmarking Multi-Turn Agentic Table Question Answering in the Wild. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:48385-48418 Available from https://proceedings.mlr.press/v306/huang26cf.html.

Related Material