Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design

Minghao Guo, Wojciech Matusik
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:38131-38212, 2026.

Abstract

Neural architecture design lacks first principles: innovations are discovered empirically and justified post-hoc, with no systematic way to diagnose why an architecture fails or derive what repair will succeed. We introduce the Axiomatic Atlas, encoding requirements as composable axioms over graph connectivity, operator contracts, numerical stability, and information preservation. Given an operator library and wiring conventions, the Atlas constructs certificates lower-bounding output variation via min-cut analysis and diagnoses failures by locating axiom violations. Crucially, the framework is prescriptive: each violation implies a targeted repair, reducing architecture design to constraint satisfaction. We prove variation bounds under exact and finite-precision arithmetic, enabling modular verification across transformers, MoEs, SSMs, and GNNs. Four Atlas-derived interventions validate the approach: +46 percentage points on GNN bottlenecks, $3\times$ robustness to MoE quantization, 83% gap closure with adaptive expert budgets, and 0%$\to$100% retrieval via orthogonal keys—each against matched negative controls.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-guo26n, title = {Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design}, author = {Guo, Minghao and Matusik, Wojciech}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {38131--38212}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/guo26n/guo26n.pdf}, url = {https://proceedings.mlr.press/v306/guo26n.html}, abstract = {Neural architecture design lacks first principles: innovations are discovered empirically and justified post-hoc, with no systematic way to diagnose why an architecture fails or derive what repair will succeed. We introduce the Axiomatic Atlas, encoding requirements as composable axioms over graph connectivity, operator contracts, numerical stability, and information preservation. Given an operator library and wiring conventions, the Atlas constructs certificates lower-bounding output variation via min-cut analysis and diagnoses failures by locating axiom violations. Crucially, the framework is prescriptive: each violation implies a targeted repair, reducing architecture design to constraint satisfaction. We prove variation bounds under exact and finite-precision arithmetic, enabling modular verification across transformers, MoEs, SSMs, and GNNs. Four Atlas-derived interventions validate the approach: +46 percentage points on GNN bottlenecks, $3\times$ robustness to MoE quantization, 83% gap closure with adaptive expert budgets, and 0%$\to$100% retrieval via orthogonal keys—each against matched negative controls.} }
Endnote
%0 Conference Paper %T Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design %A Minghao Guo %A Wojciech Matusik %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-guo26n %I PMLR %P 38131--38212 %U https://proceedings.mlr.press/v306/guo26n.html %V 306 %X Neural architecture design lacks first principles: innovations are discovered empirically and justified post-hoc, with no systematic way to diagnose why an architecture fails or derive what repair will succeed. We introduce the Axiomatic Atlas, encoding requirements as composable axioms over graph connectivity, operator contracts, numerical stability, and information preservation. Given an operator library and wiring conventions, the Atlas constructs certificates lower-bounding output variation via min-cut analysis and diagnoses failures by locating axiom violations. Crucially, the framework is prescriptive: each violation implies a targeted repair, reducing architecture design to constraint satisfaction. We prove variation bounds under exact and finite-precision arithmetic, enabling modular verification across transformers, MoEs, SSMs, and GNNs. Four Atlas-derived interventions validate the approach: +46 percentage points on GNN bottlenecks, $3\times$ robustness to MoE quantization, 83% gap closure with adaptive expert budgets, and 0%$\to$100% retrieval via orthogonal keys—each against matched negative controls.
APA
Guo, M. & Matusik, W.. (2026). Axiomatic Atlas: A Prescriptive Framework for Neural Architecture Design. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:38131-38212 Available from https://proceedings.mlr.press/v306/guo26n.html.

Related Material