ProbeLLM: Automating Principled Diagnosis of LLM Failures

Yue Huang, Zhengzhe Jiang, Yuchen Ma, Yu Jiang, Xiangqi Wang, Yujun Zhou, Yuexing Hao, Kehan Guo, Pin-Yu Chen, Marzyeh Ghassemi, Stefan Feuerriegel, Xiangliang Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:46721-46751, 2026.

Abstract

Understanding how and why large language models (LLMs) fail is becoming a central challenge as models rapidly evolve and static evaluations fall behind. While automated probing has been enabled by dynamic test generation, existing approaches often discover isolated failure cases, lack principled control over exploration, and provide limited insight into the underlying structure of model weaknesses. We propose ProbeLLM, a benchmark-agnostic automated probing framework that elevates weakness discovery from individual failures to structured failure modes. ProbeLLM formulates probing as a hierarchical Monte Carlo Tree Search, explicitly allocating limited probing budgets between global exploration of new failure regions and local refinement of recurring error patterns. By restricting probing to verifiable test cases and leveraging tool-augmented generation and verification, ProbeLLM grounds failure discovery in reliable evidence. Discovered failures are further consolidated into interpretable failure modes via failure-aware embeddings and boundary-aware induction. Across diverse benchmarks and LLMs, ProbeLLM reveals substantially broader, cleaner, and more fine-grained failure landscapes than static benchmarks and prior automated methods, supporting a shift from case-centric evaluation toward principled weakness discovery

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-huang26r, title = {{P}robe{LLM}: Automating Principled Diagnosis of {LLM} Failures}, author = {Huang, Yue and Jiang, Zhengzhe and Ma, Yuchen and Jiang, Yu and Wang, Xiangqi and Zhou, Yujun and Hao, Yuexing and Guo, Kehan and Chen, Pin-Yu and Ghassemi, Marzyeh and Feuerriegel, Stefan and Zhang, Xiangliang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {46721--46751}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/huang26r/huang26r.pdf}, url = {https://proceedings.mlr.press/v306/huang26r.html}, abstract = {Understanding how and why large language models (LLMs) fail is becoming a central challenge as models rapidly evolve and static evaluations fall behind. While automated probing has been enabled by dynamic test generation, existing approaches often discover isolated failure cases, lack principled control over exploration, and provide limited insight into the underlying structure of model weaknesses. We propose ProbeLLM, a benchmark-agnostic automated probing framework that elevates weakness discovery from individual failures to structured failure modes. ProbeLLM formulates probing as a hierarchical Monte Carlo Tree Search, explicitly allocating limited probing budgets between global exploration of new failure regions and local refinement of recurring error patterns. By restricting probing to verifiable test cases and leveraging tool-augmented generation and verification, ProbeLLM grounds failure discovery in reliable evidence. Discovered failures are further consolidated into interpretable failure modes via failure-aware embeddings and boundary-aware induction. Across diverse benchmarks and LLMs, ProbeLLM reveals substantially broader, cleaner, and more fine-grained failure landscapes than static benchmarks and prior automated methods, supporting a shift from case-centric evaluation toward principled weakness discovery} }
Endnote
%0 Conference Paper %T ProbeLLM: Automating Principled Diagnosis of LLM Failures %A Yue Huang %A Zhengzhe Jiang %A Yuchen Ma %A Yu Jiang %A Xiangqi Wang %A Yujun Zhou %A Yuexing Hao %A Kehan Guo %A Pin-Yu Chen %A Marzyeh Ghassemi %A Stefan Feuerriegel %A Xiangliang Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-huang26r %I PMLR %P 46721--46751 %U https://proceedings.mlr.press/v306/huang26r.html %V 306 %X Understanding how and why large language models (LLMs) fail is becoming a central challenge as models rapidly evolve and static evaluations fall behind. While automated probing has been enabled by dynamic test generation, existing approaches often discover isolated failure cases, lack principled control over exploration, and provide limited insight into the underlying structure of model weaknesses. We propose ProbeLLM, a benchmark-agnostic automated probing framework that elevates weakness discovery from individual failures to structured failure modes. ProbeLLM formulates probing as a hierarchical Monte Carlo Tree Search, explicitly allocating limited probing budgets between global exploration of new failure regions and local refinement of recurring error patterns. By restricting probing to verifiable test cases and leveraging tool-augmented generation and verification, ProbeLLM grounds failure discovery in reliable evidence. Discovered failures are further consolidated into interpretable failure modes via failure-aware embeddings and boundary-aware induction. Across diverse benchmarks and LLMs, ProbeLLM reveals substantially broader, cleaner, and more fine-grained failure landscapes than static benchmarks and prior automated methods, supporting a shift from case-centric evaluation toward principled weakness discovery
APA
Huang, Y., Jiang, Z., Ma, Y., Jiang, Y., Wang, X., Zhou, Y., Hao, Y., Guo, K., Chen, P., Ghassemi, M., Feuerriegel, S. & Zhang, X.. (2026). ProbeLLM: Automating Principled Diagnosis of LLM Failures. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:46721-46751 Available from https://proceedings.mlr.press/v306/huang26r.html.

Related Material