M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling

Xudong Sun, Nutan Chen, Alexej Gossmann, Yu Xing, Matteo Wohlrapp, Emilio Dorigatti, Carla Feistner, Felix Drost, Daniele Scarcella, Lisa Helen Beer, Carsten Marr
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:5149-5157, 2025.

Abstract

A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multi-objective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-sun25f, title = {M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling}, author = {Sun, Xudong and Chen, Nutan and Gossmann, Alexej and Xing, Yu and Wohlrapp, Matteo and Dorigatti, Emilio and Feistner, Carla and Drost, Felix and Scarcella, Daniele and Beer, Lisa Helen and Marr, Carsten}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {5149--5157}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/sun25f/sun25f.pdf}, url = {https://proceedings.mlr.press/v258/sun25f.html}, abstract = {A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multi-objective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks.} }
Endnote
%0 Conference Paper %T M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling %A Xudong Sun %A Nutan Chen %A Alexej Gossmann %A Yu Xing %A Matteo Wohlrapp %A Emilio Dorigatti %A Carla Feistner %A Felix Drost %A Daniele Scarcella %A Lisa Helen Beer %A Carsten Marr %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-sun25f %I PMLR %P 5149--5157 %U https://proceedings.mlr.press/v258/sun25f.html %V 258 %X A probabilistic graphical model is proposed, modeling the joint model parameter and multiplier evolution, with a hypervolume based likelihood, promoting multi-objective descent in structural risk minimization. We address multi-objective model parameter optimization via a surrogate single objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscape. The multi-objective descent goal is dispatched hierarchically into a series of constraint optimization sub-problems with shrinking bounds according to Pareto dominance. The bound serves as setpoint for the low-level multiplier controller to schedule loss landscapes via output feedback of each loss term. Our method forms closed loop of model parameter dynamic, circumvents excessive memory requirements and extra computational burden of existing multi-objective deep learning methods, and is robust against controller hyperparameter variation, demonstrated on domain generalization tasks.
APA
Sun, X., Chen, N., Gossmann, A., Xing, Y., Wohlrapp, M., Dorigatti, E., Feistner, C., Drost, F., Scarcella, D., Beer, L.H. & Marr, C.. (2025). M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:5149-5157 Available from https://proceedings.mlr.press/v258/sun25f.html.

Related Material