Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings

Hanji Du
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:26390-26410, 2026.

Abstract

The "reversal curse" exposes a critical asymmetry in autoregressive models, where models trained on facts in one direction often fail to access the corresponding inverse relation. This work studies the phenomenon from a representation-level perspective, characterizing it as a form of chiral asymmetry between subject- and object-oriented latent states. We introduce the Chiral Transformer, a lightweight framework that encourages an involutive adjoint mapping operator $\mathcal{T}$ through contrastive regularization. At inference time, Adjoint-Induced Retrieval (AIR) uses this learned map as a structured readout over model-derived entity representations, rather than as an unconstrained autoregressive generation protocol. Empirical validation on inverse-relation benchmarks shows that this symmetry-aware retrieval setting substantially improves inverse factual access, with AIR reaching 65.07% accuracy on Fact-Inv-300. These findings support a representation-access view of the reversal curse: inverse relations may be difficult not only because of missing data, but also because standard autoregressive readout fails to expose useful latent structure.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-du26a, title = {Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings}, author = {Du, Hanji}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {26390--26410}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/du26a/du26a.pdf}, url = {https://proceedings.mlr.press/v306/du26a.html}, abstract = {The "reversal curse" exposes a critical asymmetry in autoregressive models, where models trained on facts in one direction often fail to access the corresponding inverse relation. This work studies the phenomenon from a representation-level perspective, characterizing it as a form of chiral asymmetry between subject- and object-oriented latent states. We introduce the Chiral Transformer, a lightweight framework that encourages an involutive adjoint mapping operator $\mathcal{T}$ through contrastive regularization. At inference time, Adjoint-Induced Retrieval (AIR) uses this learned map as a structured readout over model-derived entity representations, rather than as an unconstrained autoregressive generation protocol. Empirical validation on inverse-relation benchmarks shows that this symmetry-aware retrieval setting substantially improves inverse factual access, with AIR reaching 65.07% accuracy on Fact-Inv-300. These findings support a representation-access view of the reversal curse: inverse relations may be difficult not only because of missing data, but also because standard autoregressive readout fails to expose useful latent structure.} }
Endnote
%0 Conference Paper %T Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings %A Hanji Du %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-du26a %I PMLR %P 26390--26410 %U https://proceedings.mlr.press/v306/du26a.html %V 306 %X The "reversal curse" exposes a critical asymmetry in autoregressive models, where models trained on facts in one direction often fail to access the corresponding inverse relation. This work studies the phenomenon from a representation-level perspective, characterizing it as a form of chiral asymmetry between subject- and object-oriented latent states. We introduce the Chiral Transformer, a lightweight framework that encourages an involutive adjoint mapping operator $\mathcal{T}$ through contrastive regularization. At inference time, Adjoint-Induced Retrieval (AIR) uses this learned map as a structured readout over model-derived entity representations, rather than as an unconstrained autoregressive generation protocol. Empirical validation on inverse-relation benchmarks shows that this symmetry-aware retrieval setting substantially improves inverse factual access, with AIR reaching 65.07% accuracy on Fact-Inv-300. These findings support a representation-access view of the reversal curse: inverse relations may be difficult not only because of missing data, but also because standard autoregressive readout fails to expose useful latent structure.
APA
Du, H.. (2026). Chiral Symmetry Breaking in Transformers: A Group-Equivariant Framework for Addressing the Reversal Curse via Adjoint Manifold Mappings. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:26390-26410 Available from https://proceedings.mlr.press/v306/du26a.html.

Related Material