Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients

Zachary Bastiani, Mike Kirby, Jacob Hochhalter, Shandian Zhe
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:2098-2106, 2026.

Abstract

We propose a novel deep symbolic regression (DSR) approach to enhance the robustness and interpretability of data-driven mathematical expression discovery. Existing DSR methods are built on recurrent neural networks, solely guided by data fitness, and potentially meet tail barriers that can zero out the policy gradient, causing inefficient model updates. To address these issues, we design a decoder-only architecture that performs attention in the frequency domain and introduce a dual-indexed position encoding to conduct layer-wise generation. Second, we propose a Bayesian information criterion (BIC)-based reward function that can automatically adjust the trade-off between expression complexity and data fitness, without the need for explicit manual tuning. Third, we develop a ranking-based weighted policy update method that eliminates the tail barriers and enhances training effectiveness. Extensive benchmarks and systematic experiments demonstrate the advantages of our approach. We have released our implementation at \url{https://github.com/ZakBastiani/CADSR.}

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-bastiani26a, title = { Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients }, author = {Bastiani, Zachary and Kirby, Mike and Hochhalter, Jacob and Zhe, Shandian}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {2098--2106}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/bastiani26a/bastiani26a.pdf}, url = {https://proceedings.mlr.press/v300/bastiani26a.html}, abstract = { We propose a novel deep symbolic regression (DSR) approach to enhance the robustness and interpretability of data-driven mathematical expression discovery. Existing DSR methods are built on recurrent neural networks, solely guided by data fitness, and potentially meet tail barriers that can zero out the policy gradient, causing inefficient model updates. To address these issues, we design a decoder-only architecture that performs attention in the frequency domain and introduce a dual-indexed position encoding to conduct layer-wise generation. Second, we propose a Bayesian information criterion (BIC)-based reward function that can automatically adjust the trade-off between expression complexity and data fitness, without the need for explicit manual tuning. Third, we develop a ranking-based weighted policy update method that eliminates the tail barriers and enhances training effectiveness. Extensive benchmarks and systematic experiments demonstrate the advantages of our approach. We have released our implementation at \url{https://github.com/ZakBastiani/CADSR.} } }
Endnote
%0 Conference Paper %T Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients %A Zachary Bastiani %A Mike Kirby %A Jacob Hochhalter %A Shandian Zhe %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-bastiani26a %I PMLR %P 2098--2106 %U https://proceedings.mlr.press/v300/bastiani26a.html %V 300 %X We propose a novel deep symbolic regression (DSR) approach to enhance the robustness and interpretability of data-driven mathematical expression discovery. Existing DSR methods are built on recurrent neural networks, solely guided by data fitness, and potentially meet tail barriers that can zero out the policy gradient, causing inefficient model updates. To address these issues, we design a decoder-only architecture that performs attention in the frequency domain and introduce a dual-indexed position encoding to conduct layer-wise generation. Second, we propose a Bayesian information criterion (BIC)-based reward function that can automatically adjust the trade-off between expression complexity and data fitness, without the need for explicit manual tuning. Third, we develop a ranking-based weighted policy update method that eliminates the tail barriers and enhances training effectiveness. Extensive benchmarks and systematic experiments demonstrate the advantages of our approach. We have released our implementation at \url{https://github.com/ZakBastiani/CADSR.}
APA
Bastiani, Z., Kirby, M., Hochhalter, J. & Zhe, S.. (2026). Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:2098-2106 Available from https://proceedings.mlr.press/v300/bastiani26a.html.

Related Material