BIOARC: Discovering Optimal Neural Architectures for Biological Foundation Models

Yi Fang, Haoran Xu, Jiaxin Han, Sirui Ding, Yizhi Wang, Yue Wang, Xuan Wang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:29447-29482, 2026.

Abstract

Foundation models have revolutionized AI, yet biological applications often repurpose general architectures without accounting for the intrinsic structural and functional properties of distinct modalities, such as genomic and proteomic sequences. Consequently, these architectures lack the inductive biases required to capture the complex “grammars" inherent to biological data, resulting in suboptimal performance. To address this, we introduce BioArc, a framework utilizing Neural Architecture Search (NAS) to shift from intuition-driven design to automated data-driven discovery. Unlike standard NAS restricted to homogeneous spaces, BioArc navigates a heterogeneous space for open-ended composition of architectural blocks. By systematically analyzing the interplay between architecture, tokenization, and training across modalities, BioArc identifies novel hybrid architectures that surpass state-of-the-art models while being up to 25x smaller. We distill these findings into empirical design principles and validate their biological relevance, demonstrating how our designs hierarchically capture the underlying biological grammar. Additionally, we introduce an agentic framework to predict optimal architectures for new tasks. Overall, BioArc provides a data-driven methodology for developing the next generation of efficient biological foundation models and task-specific networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-fang26u, title = {{BIOARC}: Discovering Optimal Neural Architectures for Biological Foundation Models}, author = {Fang, Yi and Xu, Haoran and Han, Jiaxin and Ding, Sirui and Wang, Yizhi and Wang, Yue and Wang, Xuan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {29447--29482}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/fang26u/fang26u.pdf}, url = {https://proceedings.mlr.press/v306/fang26u.html}, abstract = {Foundation models have revolutionized AI, yet biological applications often repurpose general architectures without accounting for the intrinsic structural and functional properties of distinct modalities, such as genomic and proteomic sequences. Consequently, these architectures lack the inductive biases required to capture the complex “grammars" inherent to biological data, resulting in suboptimal performance. To address this, we introduce BioArc, a framework utilizing Neural Architecture Search (NAS) to shift from intuition-driven design to automated data-driven discovery. Unlike standard NAS restricted to homogeneous spaces, BioArc navigates a heterogeneous space for open-ended composition of architectural blocks. By systematically analyzing the interplay between architecture, tokenization, and training across modalities, BioArc identifies novel hybrid architectures that surpass state-of-the-art models while being up to 25x smaller. We distill these findings into empirical design principles and validate their biological relevance, demonstrating how our designs hierarchically capture the underlying biological grammar. Additionally, we introduce an agentic framework to predict optimal architectures for new tasks. Overall, BioArc provides a data-driven methodology for developing the next generation of efficient biological foundation models and task-specific networks.} }
Endnote
%0 Conference Paper %T BIOARC: Discovering Optimal Neural Architectures for Biological Foundation Models %A Yi Fang %A Haoran Xu %A Jiaxin Han %A Sirui Ding %A Yizhi Wang %A Yue Wang %A Xuan Wang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-fang26u %I PMLR %P 29447--29482 %U https://proceedings.mlr.press/v306/fang26u.html %V 306 %X Foundation models have revolutionized AI, yet biological applications often repurpose general architectures without accounting for the intrinsic structural and functional properties of distinct modalities, such as genomic and proteomic sequences. Consequently, these architectures lack the inductive biases required to capture the complex “grammars" inherent to biological data, resulting in suboptimal performance. To address this, we introduce BioArc, a framework utilizing Neural Architecture Search (NAS) to shift from intuition-driven design to automated data-driven discovery. Unlike standard NAS restricted to homogeneous spaces, BioArc navigates a heterogeneous space for open-ended composition of architectural blocks. By systematically analyzing the interplay between architecture, tokenization, and training across modalities, BioArc identifies novel hybrid architectures that surpass state-of-the-art models while being up to 25x smaller. We distill these findings into empirical design principles and validate their biological relevance, demonstrating how our designs hierarchically capture the underlying biological grammar. Additionally, we introduce an agentic framework to predict optimal architectures for new tasks. Overall, BioArc provides a data-driven methodology for developing the next generation of efficient biological foundation models and task-specific networks.
APA
Fang, Y., Xu, H., Han, J., Ding, S., Wang, Y., Wang, Y. & Wang, X.. (2026). BIOARC: Discovering Optimal Neural Architectures for Biological Foundation Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:29447-29482 Available from https://proceedings.mlr.press/v306/fang26u.html.

Related Material