Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control

Julian Skifstad, Xinyue Annie Yang, Glen Chou
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:113885-113919, 2026.

Abstract

Inference-time LLM alignment methods, particularly activation steering, offer an alternative to fine-tuning by directly modifying activations during generation. Existing methods, however, often rely on non-anticipative interventions that ignore how perturbations propagate through transformer layers and lack online error feedback, resulting in suboptimal, open-loop control. To address this, we show empirically that layer-wise dynamics across multiple LLM architectures and scales are well-approximated by locally-linear models, despite the nonlinear structure of transformer blocks. Exploiting this property, we model LLM inference as a linear time-varying dynamical system and adapt the classical linear quadratic regulator to compute feedback controllers using layer-wise Jacobians, steering activations toward desired semantic setpoints in closed-loop with minimal computational overhead and no offline training. We also derive theoretical bounds on setpoint tracking error, enabling formal guarantees on steering performance. Using a novel adaptive semantic feature setpoint signal, our method yields robust, fine-grained behavior control across models, scales, and tasks, including state-of-the-art modulation of toxicity, truthfulness, refusal, and arbitrary concepts, surpassing baseline steering methods.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-skifstad26a, title = {Local Linearity of {LLM}s Enables Activation Steering via Model-Based Linear Optimal Control}, author = {Skifstad, Julian and Yang, Xinyue Annie and Chou, Glen}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {113885--113919}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/skifstad26a/skifstad26a.pdf}, url = {https://proceedings.mlr.press/v306/skifstad26a.html}, abstract = {Inference-time LLM alignment methods, particularly activation steering, offer an alternative to fine-tuning by directly modifying activations during generation. Existing methods, however, often rely on non-anticipative interventions that ignore how perturbations propagate through transformer layers and lack online error feedback, resulting in suboptimal, open-loop control. To address this, we show empirically that layer-wise dynamics across multiple LLM architectures and scales are well-approximated by locally-linear models, despite the nonlinear structure of transformer blocks. Exploiting this property, we model LLM inference as a linear time-varying dynamical system and adapt the classical linear quadratic regulator to compute feedback controllers using layer-wise Jacobians, steering activations toward desired semantic setpoints in closed-loop with minimal computational overhead and no offline training. We also derive theoretical bounds on setpoint tracking error, enabling formal guarantees on steering performance. Using a novel adaptive semantic feature setpoint signal, our method yields robust, fine-grained behavior control across models, scales, and tasks, including state-of-the-art modulation of toxicity, truthfulness, refusal, and arbitrary concepts, surpassing baseline steering methods.} }
Endnote
%0 Conference Paper %T Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control %A Julian Skifstad %A Xinyue Annie Yang %A Glen Chou %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-skifstad26a %I PMLR %P 113885--113919 %U https://proceedings.mlr.press/v306/skifstad26a.html %V 306 %X Inference-time LLM alignment methods, particularly activation steering, offer an alternative to fine-tuning by directly modifying activations during generation. Existing methods, however, often rely on non-anticipative interventions that ignore how perturbations propagate through transformer layers and lack online error feedback, resulting in suboptimal, open-loop control. To address this, we show empirically that layer-wise dynamics across multiple LLM architectures and scales are well-approximated by locally-linear models, despite the nonlinear structure of transformer blocks. Exploiting this property, we model LLM inference as a linear time-varying dynamical system and adapt the classical linear quadratic regulator to compute feedback controllers using layer-wise Jacobians, steering activations toward desired semantic setpoints in closed-loop with minimal computational overhead and no offline training. We also derive theoretical bounds on setpoint tracking error, enabling formal guarantees on steering performance. Using a novel adaptive semantic feature setpoint signal, our method yields robust, fine-grained behavior control across models, scales, and tasks, including state-of-the-art modulation of toxicity, truthfulness, refusal, and arbitrary concepts, surpassing baseline steering methods.
APA
Skifstad, J., Yang, X.A. & Chou, G.. (2026). Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:113885-113919 Available from https://proceedings.mlr.press/v306/skifstad26a.html.

Related Material