Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark

Jiaxu Leng, Zhoujie Huang, Mingpi Tan, Zhanjie Wu, Xinbo Gao
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:65554-65574, 2026.

Abstract

The trustworthiness of evaluation is critical to reliable model comparison and deployment in Video Anomaly Understanding (VAU). However, existing metrics are sensitive to expression styles and normal content, and this field lacks a diagnostic benchmark to validate metric validity and robustness. To bridge this gap, we propose: (1) a Class-Guided Chain-of-Evaluation (CG-CoE) metric, which structures assessment by extracting anomalous events and matching them under a class-specific semantic tolerance boundary, thereby decoupling anomaly semantics from descriptive style; and (2) an anomaly-focused meta-evaluation benchmark with two subsets: Anomalous Event-level Annotations (AEA) for measuring the validity of reflecting VAU models’ anomaly understanding ability and Controlled Variant Pairs (CVP) with fixed anomalies for quantifying robustness to stylistic perturbations. Extensive experiments demonstrate that CG-CoE achieves SOTA validity and robustness.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-leng26a, title = {Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark}, author = {Leng, Jiaxu and Huang, Zhoujie and Tan, Mingpi and Wu, Zhanjie and Gao, Xinbo}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {65554--65574}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/leng26a/leng26a.pdf}, url = {https://proceedings.mlr.press/v306/leng26a.html}, abstract = {The trustworthiness of evaluation is critical to reliable model comparison and deployment in Video Anomaly Understanding (VAU). However, existing metrics are sensitive to expression styles and normal content, and this field lacks a diagnostic benchmark to validate metric validity and robustness. To bridge this gap, we propose: (1) a Class-Guided Chain-of-Evaluation (CG-CoE) metric, which structures assessment by extracting anomalous events and matching them under a class-specific semantic tolerance boundary, thereby decoupling anomaly semantics from descriptive style; and (2) an anomaly-focused meta-evaluation benchmark with two subsets: Anomalous Event-level Annotations (AEA) for measuring the validity of reflecting VAU models’ anomaly understanding ability and Controlled Variant Pairs (CVP) with fixed anomalies for quantifying robustness to stylistic perturbations. Extensive experiments demonstrate that CG-CoE achieves SOTA validity and robustness.} }
Endnote
%0 Conference Paper %T Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark %A Jiaxu Leng %A Zhoujie Huang %A Mingpi Tan %A Zhanjie Wu %A Xinbo Gao %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-leng26a %I PMLR %P 65554--65574 %U https://proceedings.mlr.press/v306/leng26a.html %V 306 %X The trustworthiness of evaluation is critical to reliable model comparison and deployment in Video Anomaly Understanding (VAU). However, existing metrics are sensitive to expression styles and normal content, and this field lacks a diagnostic benchmark to validate metric validity and robustness. To bridge this gap, we propose: (1) a Class-Guided Chain-of-Evaluation (CG-CoE) metric, which structures assessment by extracting anomalous events and matching them under a class-specific semantic tolerance boundary, thereby decoupling anomaly semantics from descriptive style; and (2) an anomaly-focused meta-evaluation benchmark with two subsets: Anomalous Event-level Annotations (AEA) for measuring the validity of reflecting VAU models’ anomaly understanding ability and Controlled Variant Pairs (CVP) with fixed anomalies for quantifying robustness to stylistic perturbations. Extensive experiments demonstrate that CG-CoE achieves SOTA validity and robustness.
APA
Leng, J., Huang, Z., Tan, M., Wu, Z. & Gao, X.. (2026). Towards Trustworthy Video Anomaly Understanding: A Class-Guided Chain-of-Evaluation Metric and An Anomaly-focused Meta-Benchmark. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:65554-65574 Available from https://proceedings.mlr.press/v306/leng26a.html.

Related Material