Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search

Pierre-Alexandre Kamienny, Guillaume Lample, Sylvain Lamprier, Marco Virgolin
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:15655-15668, 2023.

Abstract

Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) ones. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known SRBench benchmark.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-kamienny23a, title = {Deep Generative Symbolic Regression with {M}onte-{C}arlo-Tree-Search}, author = {Kamienny, Pierre-Alexandre and Lample, Guillaume and Lamprier, Sylvain and Virgolin, Marco}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {15655--15668}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/kamienny23a/kamienny23a.pdf}, url = {https://proceedings.mlr.press/v202/kamienny23a.html}, abstract = {Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) ones. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known SRBench benchmark.} }
Endnote
%0 Conference Paper %T Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search %A Pierre-Alexandre Kamienny %A Guillaume Lample %A Sylvain Lamprier %A Marco Virgolin %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-kamienny23a %I PMLR %P 15655--15668 %U https://proceedings.mlr.press/v202/kamienny23a.html %V 202 %X Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) ones. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known SRBench benchmark.
APA
Kamienny, P., Lample, G., Lamprier, S. & Virgolin, M.. (2023). Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:15655-15668 Available from https://proceedings.mlr.press/v202/kamienny23a.html.

Related Material