DCVLR: Data Curation for Vision Language Reasoning Competition

Oussama Elachqar, Benjamin Feuer, Neha Hulkund, Rohun Tripathi, Rohun Agrawal, Thao Nguyen, Nimrod Shabtay, Vishaal Udandarao, Xiaohan Wang, Stefan Webb, Yuhui Zhang, Sara Beery, Georgia Gkioxari, Paul Liang, Ludwig Schmidt, Saining Xie, Serena Yeung-Levy, Michael Buriek, Pavel Bushuyeu, Dimitrios Damianos, Eleni Kasoura, Vassilis Katsouros, Vikas Kumar, Igor Molybog, Georgios Paraskevopoulos, Yosub Shin, Boris Sobolev, Samuel Watson, Haoyang Xu, Emmanouil Koukoumidis
Proceedings of the NeurIPS 2023-2025 Competition Track, PMLR 276:53-68, 2026.

Abstract

At NeurIPS 2025, we presented the first data-centric, open-data and open-models competition for data curation in vision-language reasoning. The competition focused on improving visual reasoning capabilities of vision-language models (VLMs) through data curation under a fixed supervised-finetuning (SFT) configuration and budget. Submissions were evaluated by fine-tuning a fixed VLM on the curated datasets and measuring performance on reasoning benchmarks of multiple-choice visual reasoning questions spanning diverse sources. The competition provided all necessary resources, including fine-tuning scripts, evaluation code, and baselines. Additionally, as a first of competitions of this kind, organizers also ran the entire SFT and evaluation pipeline on behalf of the contestants, thereby removing a key barrier to entry present in previous data-centric competitions. The result was the largest data-centric competition to date, with 130 signups and 20 submissions. DCVLR centered data curation as the key variable for scientific progress in VLM adaptation, highlighting its importance for real-world applications such as education, biomedicine, and scientific reasoning.

Cite this Paper


BibTeX
@InProceedings{pmlr-v276-elachqar26a, title = {{DCVLR}: Data Curation for Vision Language Reasoning Competition}, author = {Elachqar, Oussama and Feuer, Benjamin and Hulkund, Neha and Tripathi, Rohun and Agrawal, Rohun and Nguyen, Thao and Shabtay, Nimrod and Udandarao, Vishaal and Wang, Xiaohan and Webb, Stefan and Zhang, Yuhui and Beery, Sara and Gkioxari, Georgia and Liang, Paul and Schmidt, Ludwig and Xie, Saining and Yeung-Levy, Serena and Buriek, Michael and Bushuyeu, Pavel and Damianos, Dimitrios and Kasoura, Eleni and Katsouros, Vassilis and Kumar, Vikas and Molybog, Igor and Paraskevopoulos, Georgios and Shin, Yosub and Sobolev, Boris and Watson, Samuel and Xu, Haoyang and Koukoumidis, Emmanouil}, booktitle = {Proceedings of the NeurIPS 2023-2025 Competition Track}, pages = {53--68}, year = {2026}, editor = {Qin, Tao and Zhang, Kun and Frellsen, Jes}, volume = {276}, series = {Proceedings of Machine Learning Research}, month = {02--07 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v276/main/assets/elachqar26a/elachqar26a.pdf}, url = {https://proceedings.mlr.press/v276/elachqar26a.html}, abstract = {At NeurIPS 2025, we presented the first data-centric, open-data and open-models competition for data curation in vision-language reasoning. The competition focused on improving visual reasoning capabilities of vision-language models (VLMs) through data curation under a fixed supervised-finetuning (SFT) configuration and budget. Submissions were evaluated by fine-tuning a fixed VLM on the curated datasets and measuring performance on reasoning benchmarks of multiple-choice visual reasoning questions spanning diverse sources. The competition provided all necessary resources, including fine-tuning scripts, evaluation code, and baselines. Additionally, as a first of competitions of this kind, organizers also ran the entire SFT and evaluation pipeline on behalf of the contestants, thereby removing a key barrier to entry present in previous data-centric competitions. The result was the largest data-centric competition to date, with 130 signups and 20 submissions. DCVLR centered data curation as the key variable for scientific progress in VLM adaptation, highlighting its importance for real-world applications such as education, biomedicine, and scientific reasoning.} }
Endnote
%0 Conference Paper %T DCVLR: Data Curation for Vision Language Reasoning Competition %A Oussama Elachqar %A Benjamin Feuer %A Neha Hulkund %A Rohun Tripathi %A Rohun Agrawal %A Thao Nguyen %A Nimrod Shabtay %A Vishaal Udandarao %A Xiaohan Wang %A Stefan Webb %A Yuhui Zhang %A Sara Beery %A Georgia Gkioxari %A Paul Liang %A Ludwig Schmidt %A Saining Xie %A Serena Yeung-Levy %A Michael Buriek %A Pavel Bushuyeu %A Dimitrios Damianos %A Eleni Kasoura %A Vassilis Katsouros %A Vikas Kumar %A Igor Molybog %A Georgios Paraskevopoulos %A Yosub Shin %A Boris Sobolev %A Samuel Watson %A Haoyang Xu %A Emmanouil Koukoumidis %B Proceedings of the NeurIPS 2023-2025 Competition Track %C Proceedings of Machine Learning Research %D 2026 %E Tao Qin %E Kun Zhang %E Jes Frellsen %F pmlr-v276-elachqar26a %I PMLR %P 53--68 %U https://proceedings.mlr.press/v276/elachqar26a.html %V 276 %X At NeurIPS 2025, we presented the first data-centric, open-data and open-models competition for data curation in vision-language reasoning. The competition focused on improving visual reasoning capabilities of vision-language models (VLMs) through data curation under a fixed supervised-finetuning (SFT) configuration and budget. Submissions were evaluated by fine-tuning a fixed VLM on the curated datasets and measuring performance on reasoning benchmarks of multiple-choice visual reasoning questions spanning diverse sources. The competition provided all necessary resources, including fine-tuning scripts, evaluation code, and baselines. Additionally, as a first of competitions of this kind, organizers also ran the entire SFT and evaluation pipeline on behalf of the contestants, thereby removing a key barrier to entry present in previous data-centric competitions. The result was the largest data-centric competition to date, with 130 signups and 20 submissions. DCVLR centered data curation as the key variable for scientific progress in VLM adaptation, highlighting its importance for real-world applications such as education, biomedicine, and scientific reasoning.
APA
Elachqar, O., Feuer, B., Hulkund, N., Tripathi, R., Agrawal, R., Nguyen, T., Shabtay, N., Udandarao, V., Wang, X., Webb, S., Zhang, Y., Beery, S., Gkioxari, G., Liang, P., Schmidt, L., Xie, S., Yeung-Levy, S., Buriek, M., Bushuyeu, P., Damianos, D., Kasoura, E., Katsouros, V., Kumar, V., Molybog, I., Paraskevopoulos, G., Shin, Y., Sobolev, B., Watson, S., Xu, H. & Koukoumidis, E.. (2026). DCVLR: Data Curation for Vision Language Reasoning Competition. Proceedings of the NeurIPS 2023-2025 Competition Track, in Proceedings of Machine Learning Research 276:53-68 Available from https://proceedings.mlr.press/v276/elachqar26a.html.

Related Material