The Perception–Physics Paradox: Probing Scientific Alignment with TC-Bench

Dingling Yao, Andrea Polesello, Adeel Pervez, Caroline Muller, Francesco Locatello
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:147590-147616, 2026.

Abstract

While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invariants, making certain perception-based out-of-distribution accuracy a poor proxy for scientific utility. As a result, models may look correct without reasoning correctly—a discrepancy we term the Perception–Physics Paradox. To address this gap, we introduce Scientific Alignment as an implicit objective for representation learning in scientific domains. We study a principled, testable aspect of scientific alignment through Structural Isomorphism, which requires latent representations to uniquely identify physical systems up to a linear reparameterization. This perspective induces a hierarchy of necessary conditions and yields a systematic probing protocol for physical and causal interpretability. To operationalize this framework, we release TC-Bench, a foundational global dataset and automated construction pipeline for tropical cyclone research, and show that current VFMs rely on visual shortcuts that collapse in extreme regimes, indicating that scientific alignment does not arise as a natural byproduct of visual scaling alone.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-yao26k, title = {The Perception–Physics Paradox: Probing Scientific Alignment with {TC}-Bench}, author = {Yao, Dingling and Polesello, Andrea and Pervez, Adeel and Muller, Caroline and Locatello, Francesco}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {147590--147616}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/yao26k/yao26k.pdf}, url = {https://proceedings.mlr.press/v306/yao26k.html}, abstract = {While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invariants, making certain perception-based out-of-distribution accuracy a poor proxy for scientific utility. As a result, models may look correct without reasoning correctly—a discrepancy we term the Perception–Physics Paradox. To address this gap, we introduce Scientific Alignment as an implicit objective for representation learning in scientific domains. We study a principled, testable aspect of scientific alignment through Structural Isomorphism, which requires latent representations to uniquely identify physical systems up to a linear reparameterization. This perspective induces a hierarchy of necessary conditions and yields a systematic probing protocol for physical and causal interpretability. To operationalize this framework, we release TC-Bench, a foundational global dataset and automated construction pipeline for tropical cyclone research, and show that current VFMs rely on visual shortcuts that collapse in extreme regimes, indicating that scientific alignment does not arise as a natural byproduct of visual scaling alone.} }
Endnote
%0 Conference Paper %T The Perception–Physics Paradox: Probing Scientific Alignment with TC-Bench %A Dingling Yao %A Andrea Polesello %A Adeel Pervez %A Caroline Muller %A Francesco Locatello %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-yao26k %I PMLR %P 147590--147616 %U https://proceedings.mlr.press/v306/yao26k.html %V 306 %X While Vision Foundation Models (VFMs) excel at predictive tasks on satellite imagery, their performance can arise from visual correlations rather than underlying structural invariants, making certain perception-based out-of-distribution accuracy a poor proxy for scientific utility. As a result, models may look correct without reasoning correctly—a discrepancy we term the Perception–Physics Paradox. To address this gap, we introduce Scientific Alignment as an implicit objective for representation learning in scientific domains. We study a principled, testable aspect of scientific alignment through Structural Isomorphism, which requires latent representations to uniquely identify physical systems up to a linear reparameterization. This perspective induces a hierarchy of necessary conditions and yields a systematic probing protocol for physical and causal interpretability. To operationalize this framework, we release TC-Bench, a foundational global dataset and automated construction pipeline for tropical cyclone research, and show that current VFMs rely on visual shortcuts that collapse in extreme regimes, indicating that scientific alignment does not arise as a natural byproduct of visual scaling alone.
APA
Yao, D., Polesello, A., Pervez, A., Muller, C. & Locatello, F.. (2026). The Perception–Physics Paradox: Probing Scientific Alignment with TC-Bench. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:147590-147616 Available from https://proceedings.mlr.press/v306/yao26k.html.

Related Material