Lookahead-GCG: Improving Universal Multi-Model Optimization-Based Jailbreaking Attacks via Stochastic Nesterov Optimization

Rong Feng, Haohan Zhao, Shiqin Tang, Geng Liu, Song Lai, Meng Wang, Shuxin Zhuang, Yuqi Zha, Changyi Ma, Runsheng Yu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:30579-30592, 2026.

Abstract

Transferable jailbreaking attacks enable red-teaming of black-box large language models by optimizing adversarial prompts on open-source surrogates. A natural approach to improve transferability is multi-model training—optimizing against multiple source models simultaneously. Yet this approach has been largely abandoned, as it yields only marginal gains with standard optimizers. We argue the root cause is poor generalization: standard gradient descent lacks stability when aggregating gradients from diverse models. Since GCG and its variants implicitly perform SGD in discrete token space, they inherit this instability in multi-model settings. We address this with Lookahead-GCG, which combines: (1) Stochastic Nesterov Accelerated Gradient (SNAG), whose lookahead mechanism reduces sensitivity to individual gradient updates, providing stability for multi-model optimization; (2) embedding-space momentum accumulation, which enables SNAG in discrete token optimization; and (3) maximally distant initialization, which exploits SNAG’s improved generalization by starting from a universally beneficial point. Experiments show our method achieves 50.37% ASR on open-source and 34.03% on closed-source LLMs, outperforming GCG and TransferAttack with +11.78% gains from multi-model optimization.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-feng26y, title = {Lookahead-{GCG}: Improving Universal Multi-Model Optimization-Based Jailbreaking Attacks via Stochastic {N}esterov Optimization}, author = {Feng, Rong and Zhao, Haohan and Tang, Shiqin and Liu, Geng and Lai, Song and Wang, Meng and Zhuang, Shuxin and Zha, Yuqi and Ma, Changyi and Yu, Runsheng}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {30579--30592}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/feng26y/feng26y.pdf}, url = {https://proceedings.mlr.press/v306/feng26y.html}, abstract = {Transferable jailbreaking attacks enable red-teaming of black-box large language models by optimizing adversarial prompts on open-source surrogates. A natural approach to improve transferability is multi-model training—optimizing against multiple source models simultaneously. Yet this approach has been largely abandoned, as it yields only marginal gains with standard optimizers. We argue the root cause is poor generalization: standard gradient descent lacks stability when aggregating gradients from diverse models. Since GCG and its variants implicitly perform SGD in discrete token space, they inherit this instability in multi-model settings. We address this with Lookahead-GCG, which combines: (1) Stochastic Nesterov Accelerated Gradient (SNAG), whose lookahead mechanism reduces sensitivity to individual gradient updates, providing stability for multi-model optimization; (2) embedding-space momentum accumulation, which enables SNAG in discrete token optimization; and (3) maximally distant initialization, which exploits SNAG’s improved generalization by starting from a universally beneficial point. Experiments show our method achieves 50.37% ASR on open-source and 34.03% on closed-source LLMs, outperforming GCG and TransferAttack with +11.78% gains from multi-model optimization.} }
Endnote
%0 Conference Paper %T Lookahead-GCG: Improving Universal Multi-Model Optimization-Based Jailbreaking Attacks via Stochastic Nesterov Optimization %A Rong Feng %A Haohan Zhao %A Shiqin Tang %A Geng Liu %A Song Lai %A Meng Wang %A Shuxin Zhuang %A Yuqi Zha %A Changyi Ma %A Runsheng Yu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-feng26y %I PMLR %P 30579--30592 %U https://proceedings.mlr.press/v306/feng26y.html %V 306 %X Transferable jailbreaking attacks enable red-teaming of black-box large language models by optimizing adversarial prompts on open-source surrogates. A natural approach to improve transferability is multi-model training—optimizing against multiple source models simultaneously. Yet this approach has been largely abandoned, as it yields only marginal gains with standard optimizers. We argue the root cause is poor generalization: standard gradient descent lacks stability when aggregating gradients from diverse models. Since GCG and its variants implicitly perform SGD in discrete token space, they inherit this instability in multi-model settings. We address this with Lookahead-GCG, which combines: (1) Stochastic Nesterov Accelerated Gradient (SNAG), whose lookahead mechanism reduces sensitivity to individual gradient updates, providing stability for multi-model optimization; (2) embedding-space momentum accumulation, which enables SNAG in discrete token optimization; and (3) maximally distant initialization, which exploits SNAG’s improved generalization by starting from a universally beneficial point. Experiments show our method achieves 50.37% ASR on open-source and 34.03% on closed-source LLMs, outperforming GCG and TransferAttack with +11.78% gains from multi-model optimization.
APA
Feng, R., Zhao, H., Tang, S., Liu, G., Lai, S., Wang, M., Zhuang, S., Zha, Y., Ma, C. & Yu, R.. (2026). Lookahead-GCG: Improving Universal Multi-Model Optimization-Based Jailbreaking Attacks via Stochastic Nesterov Optimization. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:30579-30592 Available from https://proceedings.mlr.press/v306/feng26y.html.

Related Material