Post-Training Language Models for Crosslingual Consistency

Tianyu Liu, Jirui Qi, Mrinmaya Sachan, Ryan Cotterell, Raquel Fernández, Arianna Bisazza
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:77970-77994, 2026.

Abstract

Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model’s response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback–Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-liu26dd, title = {Post-Training Language Models for Crosslingual Consistency}, author = {Liu, Tianyu and Qi, Jirui and Sachan, Mrinmaya and Cotterell, Ryan and Fern\'{a}ndez, Raquel and Bisazza, Arianna}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {77970--77994}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/liu26dd/liu26dd.pdf}, url = {https://proceedings.mlr.press/v306/liu26dd.html}, abstract = {Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model’s response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback–Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.} }
Endnote
%0 Conference Paper %T Post-Training Language Models for Crosslingual Consistency %A Tianyu Liu %A Jirui Qi %A Mrinmaya Sachan %A Ryan Cotterell %A Raquel Fernández %A Arianna Bisazza %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-liu26dd %I PMLR %P 77970--77994 %U https://proceedings.mlr.press/v306/liu26dd.html %V 306 %X Language models often respond inconsistently to translation-equivalent prompts across languages, undermining the reliability of multilingual systems. To quantify this, we give an information-theoretic definition of crosslingual consistency as a divergence bound between a model’s response distribution and its round-trip pushforward across languages. We then introduce penalized consistency optimization (PCO), a post-training procedure that couples this divergence with a Kullback–Leibler penalty to a fixed reference language model. Because direct optimization of PCO requires expensive on-policy roll-outs, we propose a tractable surrogate, direct consistency optimization (DCO), which can be optimized off-policy. Across diverse language models and 26 languages, DCO significantly improves crosslingual consistency, outperforms existing methods, and enables targeted alignment of low-resource languages.
APA
Liu, T., Qi, J., Sachan, M., Cotterell, R., Fernández, R. & Bisazza, A.. (2026). Post-Training Language Models for Crosslingual Consistency. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:77970-77994 Available from https://proceedings.mlr.press/v306/liu26dd.html.

Related Material