Scalable Model-Assisted Multi-Target Estimation in Large Image Collections

Max Hamilton, Jinlin Lai, Daniel Sheldon, Subhransu Maji
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:1931-1946, 2026.

Abstract

Computer vision models are increasingly used as measurement tools to estimate population-level quantities from large image collections, but prediction errors introduce bias and the resulting estimates lack statistical guarantees required in scientific applications. Prior work uses a Monte Carlo framework to combine model predictions with ground-truth annotations by sampling some images for humans to label and is able to provide unbiased estimates with controllable accuracy, but primarily addresses single-scalar estimation. We study the more general problem of multi-target estimation, where many quantities (e.g., class counts or proportions) must be estimated simultaneously, and adapt sampling and estimation strategies from survey sampling to this setting. Evaluations on five detection and segmentation datasets with 7–80 classes show that importance sampling excels with moderate annotation budgets or fewer targets, whereas uniform sampling with control variates is superior when estimating many targets or operating with minimal labels. Additionally, a subset-based ratio estimator remains highly competitive across all regimes. Ultimately, our framework effectively combines biased model predictions and limited human labels into rigorous scientific measurements.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-hamilton26a, title = {Scalable Model-Assisted Multi-Target Estimation in Large Image Collections}, author = {Hamilton, Max and Lai, Jinlin and Sheldon, Daniel and Maji, Subhransu}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {1931--1946}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/hamilton26a/hamilton26a.pdf}, url = {https://proceedings.mlr.press/v337/hamilton26a.html}, abstract = {Computer vision models are increasingly used as measurement tools to estimate population-level quantities from large image collections, but prediction errors introduce bias and the resulting estimates lack statistical guarantees required in scientific applications. Prior work uses a Monte Carlo framework to combine model predictions with ground-truth annotations by sampling some images for humans to label and is able to provide unbiased estimates with controllable accuracy, but primarily addresses single-scalar estimation. We study the more general problem of multi-target estimation, where many quantities (e.g., class counts or proportions) must be estimated simultaneously, and adapt sampling and estimation strategies from survey sampling to this setting. Evaluations on five detection and segmentation datasets with 7–80 classes show that importance sampling excels with moderate annotation budgets or fewer targets, whereas uniform sampling with control variates is superior when estimating many targets or operating with minimal labels. Additionally, a subset-based ratio estimator remains highly competitive across all regimes. Ultimately, our framework effectively combines biased model predictions and limited human labels into rigorous scientific measurements.} }
Endnote
%0 Conference Paper %T Scalable Model-Assisted Multi-Target Estimation in Large Image Collections %A Max Hamilton %A Jinlin Lai %A Daniel Sheldon %A Subhransu Maji %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-hamilton26a %I PMLR %P 1931--1946 %U https://proceedings.mlr.press/v337/hamilton26a.html %V 337 %X Computer vision models are increasingly used as measurement tools to estimate population-level quantities from large image collections, but prediction errors introduce bias and the resulting estimates lack statistical guarantees required in scientific applications. Prior work uses a Monte Carlo framework to combine model predictions with ground-truth annotations by sampling some images for humans to label and is able to provide unbiased estimates with controllable accuracy, but primarily addresses single-scalar estimation. We study the more general problem of multi-target estimation, where many quantities (e.g., class counts or proportions) must be estimated simultaneously, and adapt sampling and estimation strategies from survey sampling to this setting. Evaluations on five detection and segmentation datasets with 7–80 classes show that importance sampling excels with moderate annotation budgets or fewer targets, whereas uniform sampling with control variates is superior when estimating many targets or operating with minimal labels. Additionally, a subset-based ratio estimator remains highly competitive across all regimes. Ultimately, our framework effectively combines biased model predictions and limited human labels into rigorous scientific measurements.
APA
Hamilton, M., Lai, J., Sheldon, D. & Maji, S.. (2026). Scalable Model-Assisted Multi-Target Estimation in Large Image Collections. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:1931-1946 Available from https://proceedings.mlr.press/v337/hamilton26a.html.

Related Material