Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control

Hao Ren, Zetong Bi, Yiming Zeng, Le Zheng, Zhi Li, Zhaoliang Wan, Lu Qi, Hui Cheng
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:104364-104390, 2026.

Abstract

Diffusion models are effective for waypoint prediction in visual navigation, but standard sampling and test time guidance can produce unreliable or inefficient trajectories when updates drift off the training manifold. We propose Fisher Preserving Guidance with Outer Product Span Projection, a training-free inference method that avoids large Fisher drift associated with off-distribution actions while optimizing a task objective. Our method computes the Fisher-preserving update via a low-rank Jacobian factorization, requiring only a single backward pass per step and enabling real-time use. We further introduce Truncated Fisher Denoising Sensitivity as an uncertainty signal and use it for robust multi-sample action blending. Experiments on toy and realistic navigation benchmarks, including Maze2D with TSDF-based guidance, PushT with official Diffusion Policy weights, and visual navigation in simulation and on real robots, demonstrate consistent improvements in performance over strong diffusion-policy baselines without additional training.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-ren26a, title = {{F}isher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control}, author = {Ren, Hao and Bi, Zetong and Zeng, Yiming and Zheng, Le and Li, Zhi and Wan, Zhaoliang and Qi, Lu and Cheng, Hui}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {104364--104390}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/ren26a/ren26a.pdf}, url = {https://proceedings.mlr.press/v306/ren26a.html}, abstract = {Diffusion models are effective for waypoint prediction in visual navigation, but standard sampling and test time guidance can produce unreliable or inefficient trajectories when updates drift off the training manifold. We propose Fisher Preserving Guidance with Outer Product Span Projection, a training-free inference method that avoids large Fisher drift associated with off-distribution actions while optimizing a task objective. Our method computes the Fisher-preserving update via a low-rank Jacobian factorization, requiring only a single backward pass per step and enabling real-time use. We further introduce Truncated Fisher Denoising Sensitivity as an uncertainty signal and use it for robust multi-sample action blending. Experiments on toy and realistic navigation benchmarks, including Maze2D with TSDF-based guidance, PushT with official Diffusion Policy weights, and visual navigation in simulation and on real robots, demonstrate consistent improvements in performance over strong diffusion-policy baselines without additional training.} }
Endnote
%0 Conference Paper %T Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control %A Hao Ren %A Zetong Bi %A Yiming Zeng %A Le Zheng %A Zhi Li %A Zhaoliang Wan %A Lu Qi %A Hui Cheng %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-ren26a %I PMLR %P 104364--104390 %U https://proceedings.mlr.press/v306/ren26a.html %V 306 %X Diffusion models are effective for waypoint prediction in visual navigation, but standard sampling and test time guidance can produce unreliable or inefficient trajectories when updates drift off the training manifold. We propose Fisher Preserving Guidance with Outer Product Span Projection, a training-free inference method that avoids large Fisher drift associated with off-distribution actions while optimizing a task objective. Our method computes the Fisher-preserving update via a low-rank Jacobian factorization, requiring only a single backward pass per step and enabling real-time use. We further introduce Truncated Fisher Denoising Sensitivity as an uncertainty signal and use it for robust multi-sample action blending. Experiments on toy and realistic navigation benchmarks, including Maze2D with TSDF-based guidance, PushT with official Diffusion Policy weights, and visual navigation in simulation and on real robots, demonstrate consistent improvements in performance over strong diffusion-policy baselines without additional training.
APA
Ren, H., Bi, Z., Zeng, Y., Zheng, L., Li, Z., Wan, Z., Qi, L. & Cheng, H.. (2026). Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:104364-104390 Available from https://proceedings.mlr.press/v306/ren26a.html.

Related Material