ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models

Jiahui Guang, Haiyan Wang, Yingjie Zhu, Cuiyun Gao, Jing Li, Di Shao, Zhaoquan Gu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:37657-37679, 2026.

Abstract

Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Existing methods typically evaluate unlearning effectiveness based on output deviations, while overlooking the generation quality after unlearning. This can easily lead to hallucinated or rigid responses, thereby affecting the usability and safety of the unlearned model. To address this issue, we propose ASRU, a controllable multimodal unlearning framework that incorporates generation quality as a core evaluation objective. ASRU first induces initial refusal behavior through activation redirection, and then optimizes fine-grained refusal boundaries using a customized reward function, thereby achieving a better trade-off between target knowledge unlearning and model utility. Experiments on Qwen3-VL show that ASRU significantly improves unlearning effectiveness (+24.6%) on average and generation quality (5.8$\times$) on average while effectively preserving model utility, using only a small amount of retained supervision data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-guang26a, title = {{ASRU}: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models}, author = {Guang, Jiahui and Wang, Haiyan and Zhu, Yingjie and Gao, Cuiyun and Li, Jing and Shao, Di and Gu, Zhaoquan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {37657--37679}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/guang26a/guang26a.pdf}, url = {https://proceedings.mlr.press/v306/guang26a.html}, abstract = {Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Existing methods typically evaluate unlearning effectiveness based on output deviations, while overlooking the generation quality after unlearning. This can easily lead to hallucinated or rigid responses, thereby affecting the usability and safety of the unlearned model. To address this issue, we propose ASRU, a controllable multimodal unlearning framework that incorporates generation quality as a core evaluation objective. ASRU first induces initial refusal behavior through activation redirection, and then optimizes fine-grained refusal boundaries using a customized reward function, thereby achieving a better trade-off between target knowledge unlearning and model utility. Experiments on Qwen3-VL show that ASRU significantly improves unlearning effectiveness (+24.6%) on average and generation quality (5.8$\times$) on average while effectively preserving model utility, using only a small amount of retained supervision data.} }
Endnote
%0 Conference Paper %T ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models %A Jiahui Guang %A Haiyan Wang %A Yingjie Zhu %A Cuiyun Gao %A Jing Li %A Di Shao %A Zhaoquan Gu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-guang26a %I PMLR %P 37657--37679 %U https://proceedings.mlr.press/v306/guang26a.html %V 306 %X Multimodal large language models (MLLMs) may memorize sensitive cross-modal information during pretraining, making machine unlearning (MU) crucial. Existing methods typically evaluate unlearning effectiveness based on output deviations, while overlooking the generation quality after unlearning. This can easily lead to hallucinated or rigid responses, thereby affecting the usability and safety of the unlearned model. To address this issue, we propose ASRU, a controllable multimodal unlearning framework that incorporates generation quality as a core evaluation objective. ASRU first induces initial refusal behavior through activation redirection, and then optimizes fine-grained refusal boundaries using a customized reward function, thereby achieving a better trade-off between target knowledge unlearning and model utility. Experiments on Qwen3-VL show that ASRU significantly improves unlearning effectiveness (+24.6%) on average and generation quality (5.8$\times$) on average while effectively preserving model utility, using only a small amount of retained supervision data.
APA
Guang, J., Wang, H., Zhu, Y., Gao, C., Li, J., Shao, D. & Gu, Z.. (2026). ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:37657-37679 Available from https://proceedings.mlr.press/v306/guang26a.html.

Related Material