tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)

Junhua Zeng, Chao Li, Zhun Sun, Qibin Zhao, Guoxu Zhou
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:58329-58347, 2024.

Abstract

Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods. Our code is available at https://github.com/ChaoLiAtRIKEN/tngps.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-zeng24b, title = {tn{GPS}: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models ({LLM}s)}, author = {Zeng, Junhua and Li, Chao and Sun, Zhun and Zhao, Qibin and Zhou, Guoxu}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {58329--58347}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/zeng24b/zeng24b.pdf}, url = {https://proceedings.mlr.press/v235/zeng24b.html}, abstract = {Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods. Our code is available at https://github.com/ChaoLiAtRIKEN/tngps.} }
Endnote
%0 Conference Paper %T tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs) %A Junhua Zeng %A Chao Li %A Zhun Sun %A Qibin Zhao %A Guoxu Zhou %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-zeng24b %I PMLR %P 58329--58347 %U https://proceedings.mlr.press/v235/zeng24b.html %V 235 %X Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods. Our code is available at https://github.com/ChaoLiAtRIKEN/tngps.
APA
Zeng, J., Li, C., Sun, Z., Zhao, Q. & Zhou, G.. (2024). tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs). Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:58329-58347 Available from https://proceedings.mlr.press/v235/zeng24b.html.

Related Material