Reliable Cultural Knowledge Preservation in Multilingual LLMs through Model Merging

Hoang Quan Nguyen, Nhut Huy Pham, Maziyar Pahani, Johanna Björklund, Xuan-Son Vu
Reliable and Trustworthy Artificial Intelligence 2025, PMLR 310:59-66, 2025.

Abstract

We introduce a reliable approach for enhancing multilingual language models that preserves cultural knowledge while improving reasoning capabilities, focusing on low-resource languages. Using Qwen as a base model, we demonstrate that trust-aware model merging can verifiably improve performance without compromising cultural understanding. Our proposed approach achieves quantifiable improvements in both reasoning tasks and cultural benchmarks while maintaining computational efficiency. Results on Vietnamese and Arabic language tasks show consistent performance gains while preserving cultural knowledge, offering a reliable path for developing trustworthy multilingual AI systems. Our models are available at github.com/WARA-ML/waraml-mini-brains.

Cite this Paper

BibTeX
@InProceedings{pmlr-v310-nguyen25b, title = {Reliable Cultural Knowledge Preservation in Multilingual LLMs through Model Merging}, author = {Nguyen, Hoang Quan and Pham, Nhut Huy and Pahani, Maziyar and Bj{\"o}rklund, Johanna and Vu, Xuan-Son}, booktitle = {Reliable and Trustworthy Artificial Intelligence 2025}, pages = {59--66}, year = {2025}, editor = {Nguyen, Hoang D. and Le, Duc-Trong and Björklund, Johanna and Vu, Xuan-Son}, volume = {310}, series = {Proceedings of Machine Learning Research}, month = {12 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v310/main/assets/nguyen25b/nguyen25b.pdf}, url = {https://proceedings.mlr.press/v310/nguyen25b.html}, abstract = {We introduce a reliable approach for enhancing multilingual language models that preserves cultural knowledge while improving reasoning capabilities, focusing on low-resource languages. Using Qwen as a base model, we demonstrate that trust-aware model merging can verifiably improve performance without compromising cultural understanding. Our proposed approach achieves quantifiable improvements in both reasoning tasks and cultural benchmarks while maintaining computational efficiency. Results on Vietnamese and Arabic language tasks show consistent performance gains while preserving cultural knowledge, offering a reliable path for developing trustworthy multilingual AI systems. Our models are available at github.com/WARA-ML/waraml-mini-brains.} }
Endnote
%0 Conference Paper %T Reliable Cultural Knowledge Preservation in Multilingual LLMs through Model Merging %A Hoang Quan Nguyen %A Nhut Huy Pham %A Maziyar Pahani %A Johanna Björklund %A Xuan-Son Vu %B Reliable and Trustworthy Artificial Intelligence 2025 %C Proceedings of Machine Learning Research %D 2025 %E Hoang D. Nguyen %E Duc-Trong Le %E Johanna Björklund %E Xuan-Son Vu %F pmlr-v310-nguyen25b %I PMLR %P 59--66 %U https://proceedings.mlr.press/v310/nguyen25b.html %V 310 %X We introduce a reliable approach for enhancing multilingual language models that preserves cultural knowledge while improving reasoning capabilities, focusing on low-resource languages. Using Qwen as a base model, we demonstrate that trust-aware model merging can verifiably improve performance without compromising cultural understanding. Our proposed approach achieves quantifiable improvements in both reasoning tasks and cultural benchmarks while maintaining computational efficiency. Results on Vietnamese and Arabic language tasks show consistent performance gains while preserving cultural knowledge, offering a reliable path for developing trustworthy multilingual AI systems. Our models are available at github.com/WARA-ML/waraml-mini-brains.
APA
Nguyen, H.Q., Pham, N.H., Pahani, M., Björklund, J. & Vu, X.. (2025). Reliable Cultural Knowledge Preservation in Multilingual LLMs through Model Merging. Reliable and Trustworthy Artificial Intelligence 2025, in Proceedings of Machine Learning Research 310:59-66 Available from https://proceedings.mlr.press/v310/nguyen25b.html.

Related Material