Disentangling Consensus and Value-Specific Representations for Controllable Pluralistic Value Alignment in LLMs

Jiankui Zhou, Jing Yao, Xiaoyuan Yi, Peng Zhang, Ning Gu, Zhan Hu, Xing Xie, Tun Lu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:166196-166220, 2026.

Abstract

With the widespread deployment of large language models (LLMs), aligning model outputs with pluralistic human values has become an important research problem. Recent approaches that train task-specific experts and merge them through parameter aggregation have shown promise for pluralistic alignment. However, these methods often overlook the intrinsic complexity of real-world value data, where multiple correlated value dimensions coexist, resulting in highly similar and entangled expert representations. Consequently, modifying the contribution of one value expert may unintentionally influence other values, limiting fine-grained controllability. To address this issue, we propose DisAlign, a model-merging framework that explicitly decomposes value representations into consensus and value-specific components using an information-geometric perspective. DisAlign first extracts a consensus anchor and subspace to capture shared structure across values, and then applies spectral decomposition to the residual representations to construct disentangled value subspaces. This design enables more precise and independent modulation of multiple values. Experiments on three datasets covering different value frameworks demonstrate that DisAlign consistently improves value disentanglement and achieves more accurate pluralistic value control compared to existing baselines. Our code is available at https://anonymous.4open.science/r/DisAlign-7F35

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zhou26bf, title = {Disentangling Consensus and Value-Specific Representations for Controllable Pluralistic Value Alignment in {LLM}s}, author = {Zhou, Jiankui and Yao, Jing and Yi, Xiaoyuan and Zhang, Peng and Gu, Ning and Hu, Zhan and Xie, Xing and Lu, Tun}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {166196--166220}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhou26bf/zhou26bf.pdf}, url = {https://proceedings.mlr.press/v306/zhou26bf.html}, abstract = {With the widespread deployment of large language models (LLMs), aligning model outputs with pluralistic human values has become an important research problem. Recent approaches that train task-specific experts and merge them through parameter aggregation have shown promise for pluralistic alignment. However, these methods often overlook the intrinsic complexity of real-world value data, where multiple correlated value dimensions coexist, resulting in highly similar and entangled expert representations. Consequently, modifying the contribution of one value expert may unintentionally influence other values, limiting fine-grained controllability. To address this issue, we propose DisAlign, a model-merging framework that explicitly decomposes value representations into consensus and value-specific components using an information-geometric perspective. DisAlign first extracts a consensus anchor and subspace to capture shared structure across values, and then applies spectral decomposition to the residual representations to construct disentangled value subspaces. This design enables more precise and independent modulation of multiple values. Experiments on three datasets covering different value frameworks demonstrate that DisAlign consistently improves value disentanglement and achieves more accurate pluralistic value control compared to existing baselines. Our code is available at https://anonymous.4open.science/r/DisAlign-7F35} }
Endnote
%0 Conference Paper %T Disentangling Consensus and Value-Specific Representations for Controllable Pluralistic Value Alignment in LLMs %A Jiankui Zhou %A Jing Yao %A Xiaoyuan Yi %A Peng Zhang %A Ning Gu %A Zhan Hu %A Xing Xie %A Tun Lu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhou26bf %I PMLR %P 166196--166220 %U https://proceedings.mlr.press/v306/zhou26bf.html %V 306 %X With the widespread deployment of large language models (LLMs), aligning model outputs with pluralistic human values has become an important research problem. Recent approaches that train task-specific experts and merge them through parameter aggregation have shown promise for pluralistic alignment. However, these methods often overlook the intrinsic complexity of real-world value data, where multiple correlated value dimensions coexist, resulting in highly similar and entangled expert representations. Consequently, modifying the contribution of one value expert may unintentionally influence other values, limiting fine-grained controllability. To address this issue, we propose DisAlign, a model-merging framework that explicitly decomposes value representations into consensus and value-specific components using an information-geometric perspective. DisAlign first extracts a consensus anchor and subspace to capture shared structure across values, and then applies spectral decomposition to the residual representations to construct disentangled value subspaces. This design enables more precise and independent modulation of multiple values. Experiments on three datasets covering different value frameworks demonstrate that DisAlign consistently improves value disentanglement and achieves more accurate pluralistic value control compared to existing baselines. Our code is available at https://anonymous.4open.science/r/DisAlign-7F35
APA
Zhou, J., Yao, J., Yi, X., Zhang, P., Gu, N., Hu, Z., Xie, X. & Lu, T.. (2026). Disentangling Consensus and Value-Specific Representations for Controllable Pluralistic Value Alignment in LLMs. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:166196-166220 Available from https://proceedings.mlr.press/v306/zhou26bf.html.

Related Material