ChartE$^3$: A Comprehensive Benchmark for End-to-End Chart Editing

Shuo Li, Jiajun Sun, Zhekai Wang, Xiaoran Fan, Hui Li, Dingwen Yang, Zhiheng Xi, Yijun Wang, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:70091-70112, 2026.

Abstract

Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains challenging due to the need for both fine-grained control and global structural consistency. Most existing approaches adopt pipeline-based designs, where natural language or code serves as an intermediate representation, limiting their ability to faithfully execute complex edits. We introduce ChartE$^{3}$, an End-to-End Chart Editing benchmark that directly evaluates models without relying on intermediate natural language programs or code-level supervision. ChartE$^{3}$ focuses on two complementary editing dimensions: local editing, which involves fine-grained appearance changes such as font or color adjustments, and global editing, which requires holistic, data-centric transformations including data filtering and trend line addition. ChartE$^{3}$ contains over 1,200 high-quality samples constructed via an well-designed data pipeline with human curation. Each sample is provided as a triplet of a chart image, its underlying code, and a multimodal editing instruction, enabling evaluation from both visual and structural perspectives. Extensive benchmarking of state-of-the-art multimodal large language models reveals substantial performance gaps, particularly on global editing tasks, highlighting critical limitations in current end-to-end chart editing capabilities.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-li26fr, title = {{C}hart{E}$^{3}$: A Comprehensive Benchmark for End-to-End Chart Editing}, author = {Li, Shuo and Sun, Jiajun and Wang, Zhekai and Fan, Xiaoran and Li, Hui and Yang, Dingwen and Xi, Zhiheng and Wang, Yijun and Shan, Zifei and Gui, Tao and Zhang, Qi and Huang, Xuanjing}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {70091--70112}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/li26fr/li26fr.pdf}, url = {https://proceedings.mlr.press/v306/li26fr.html}, abstract = {Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains challenging due to the need for both fine-grained control and global structural consistency. Most existing approaches adopt pipeline-based designs, where natural language or code serves as an intermediate representation, limiting their ability to faithfully execute complex edits. We introduce ChartE$^{3}$, an End-to-End Chart Editing benchmark that directly evaluates models without relying on intermediate natural language programs or code-level supervision. ChartE$^{3}$ focuses on two complementary editing dimensions: local editing, which involves fine-grained appearance changes such as font or color adjustments, and global editing, which requires holistic, data-centric transformations including data filtering and trend line addition. ChartE$^{3}$ contains over 1,200 high-quality samples constructed via an well-designed data pipeline with human curation. Each sample is provided as a triplet of a chart image, its underlying code, and a multimodal editing instruction, enabling evaluation from both visual and structural perspectives. Extensive benchmarking of state-of-the-art multimodal large language models reveals substantial performance gaps, particularly on global editing tasks, highlighting critical limitations in current end-to-end chart editing capabilities.} }
Endnote
%0 Conference Paper %T ChartE$^3$: A Comprehensive Benchmark for End-to-End Chart Editing %A Shuo Li %A Jiajun Sun %A Zhekai Wang %A Xiaoran Fan %A Hui Li %A Dingwen Yang %A Zhiheng Xi %A Yijun Wang %A Zifei Shan %A Tao Gui %A Qi Zhang %A Xuanjing Huang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-li26fr %I PMLR %P 70091--70112 %U https://proceedings.mlr.press/v306/li26fr.html %V 306 %X Charts are a fundamental visualization format for structured data analysis. Enabling end-to-end chart editing according to user intent is of great practical value, yet remains challenging due to the need for both fine-grained control and global structural consistency. Most existing approaches adopt pipeline-based designs, where natural language or code serves as an intermediate representation, limiting their ability to faithfully execute complex edits. We introduce ChartE$^{3}$, an End-to-End Chart Editing benchmark that directly evaluates models without relying on intermediate natural language programs or code-level supervision. ChartE$^{3}$ focuses on two complementary editing dimensions: local editing, which involves fine-grained appearance changes such as font or color adjustments, and global editing, which requires holistic, data-centric transformations including data filtering and trend line addition. ChartE$^{3}$ contains over 1,200 high-quality samples constructed via an well-designed data pipeline with human curation. Each sample is provided as a triplet of a chart image, its underlying code, and a multimodal editing instruction, enabling evaluation from both visual and structural perspectives. Extensive benchmarking of state-of-the-art multimodal large language models reveals substantial performance gaps, particularly on global editing tasks, highlighting critical limitations in current end-to-end chart editing capabilities.
APA
Li, S., Sun, J., Wang, Z., Fan, X., Li, H., Yang, D., Xi, Z., Wang, Y., Shan, Z., Gui, T., Zhang, Q. & Huang, X.. (2026). ChartE$^3$: A Comprehensive Benchmark for End-to-End Chart Editing. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:70091-70112 Available from https://proceedings.mlr.press/v306/li26fr.html.

Related Material