Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics

Xiaoyan Xing, Xiao Zhang, Sezer Karaoglu, Theo Gevers, Anand Bhattad
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:140189-140205, 2026.

Abstract

Image-to-image relighting requires representations that separate illumination from scene properties while preserving dense geometry, material, and photometric cues. We use this task as a probe of visual priors: unlike recognition tasks that reward invariance, relighting tests whether visual features retain the information needed for light transfer. Through a controlled generative relighting framework, we find that strong semantic encoders can degrade relighting quality, exposing a semantic–photometric trade-off between abstraction and physical fidelity. We introduce Augmented Latent Intrinsics (ALI), which balances this trade-off by fusing dense, pixel-aligned visual features into a latent-intrinsic relighting model and refining it with self-supervision on unlabeled real image pairs. ALI improves relighting quality, especially on glossy, metallic, and transparent materials, and demonstrates that generative relighting is an effective tool for quantifying what visual encoders encode about the physical world.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-xing26i, title = {Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics}, author = {Xing, Xiaoyan and Zhang, Xiao and Karaoglu, Sezer and Gevers, Theo and Bhattad, Anand}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {140189--140205}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/xing26i/xing26i.pdf}, url = {https://proceedings.mlr.press/v306/xing26i.html}, abstract = {Image-to-image relighting requires representations that separate illumination from scene properties while preserving dense geometry, material, and photometric cues. We use this task as a probe of visual priors: unlike recognition tasks that reward invariance, relighting tests whether visual features retain the information needed for light transfer. Through a controlled generative relighting framework, we find that strong semantic encoders can degrade relighting quality, exposing a semantic–photometric trade-off between abstraction and physical fidelity. We introduce Augmented Latent Intrinsics (ALI), which balances this trade-off by fusing dense, pixel-aligned visual features into a latent-intrinsic relighting model and refining it with self-supervision on unlabeled real image pairs. ALI improves relighting quality, especially on glossy, metallic, and transparent materials, and demonstrates that generative relighting is an effective tool for quantifying what visual encoders encode about the physical world.} }
Endnote
%0 Conference Paper %T Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics %A Xiaoyan Xing %A Xiao Zhang %A Sezer Karaoglu %A Theo Gevers %A Anand Bhattad %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-xing26i %I PMLR %P 140189--140205 %U https://proceedings.mlr.press/v306/xing26i.html %V 306 %X Image-to-image relighting requires representations that separate illumination from scene properties while preserving dense geometry, material, and photometric cues. We use this task as a probe of visual priors: unlike recognition tasks that reward invariance, relighting tests whether visual features retain the information needed for light transfer. Through a controlled generative relighting framework, we find that strong semantic encoders can degrade relighting quality, exposing a semantic–photometric trade-off between abstraction and physical fidelity. We introduce Augmented Latent Intrinsics (ALI), which balances this trade-off by fusing dense, pixel-aligned visual features into a latent-intrinsic relighting model and refining it with self-supervision on unlabeled real image pairs. ALI improves relighting quality, especially on glossy, metallic, and transparent materials, and demonstrates that generative relighting is an effective tool for quantifying what visual encoders encode about the physical world.
APA
Xing, X., Zhang, X., Karaoglu, S., Gevers, T. & Bhattad, A.. (2026). Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:140189-140205 Available from https://proceedings.mlr.press/v306/xing26i.html.

Related Material