Reinforcement Learning within Tree Search for Fast Macro Placement

Zijie Geng, Jie Wang, Ziyan Liu, Siyuan Xu, Zhentao Tang, Mingxuan Yuan, Jianye Hao, Yongdong Zhang, Feng Wu
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:15402-15417, 2024.

Abstract

Macro placement is a crucial step in modern chip design, and reinforcement learning (RL) has recently emerged as a promising technique for improving the placement quality. However, existing RL-based techniques are hindered by their low sample efficiency, requiring numerous online rollouts or substantial offline expert data to achieve bootstrap, which are often impractical in industrial scenarios. To address this challenge, we propose a novel sample-efficient framework, namely EfficientPlace, for fast macro placement. EfficientPlace integrates a global tree search algorithm to strategically direct the optimization process, as well as a RL agent for local policy learning to advance the tree search. Experiments on commonly used benchmarks demonstrate that EfficientPlace achieves remarkable placement quality within a short timeframe, outperforming recent state-of-the-art approaches.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-geng24b, title = {Reinforcement Learning within Tree Search for Fast Macro Placement}, author = {Geng, Zijie and Wang, Jie and Liu, Ziyan and Xu, Siyuan and Tang, Zhentao and Yuan, Mingxuan and Hao, Jianye and Zhang, Yongdong and Wu, Feng}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {15402--15417}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/geng24b/geng24b.pdf}, url = {https://proceedings.mlr.press/v235/geng24b.html}, abstract = {Macro placement is a crucial step in modern chip design, and reinforcement learning (RL) has recently emerged as a promising technique for improving the placement quality. However, existing RL-based techniques are hindered by their low sample efficiency, requiring numerous online rollouts or substantial offline expert data to achieve bootstrap, which are often impractical in industrial scenarios. To address this challenge, we propose a novel sample-efficient framework, namely EfficientPlace, for fast macro placement. EfficientPlace integrates a global tree search algorithm to strategically direct the optimization process, as well as a RL agent for local policy learning to advance the tree search. Experiments on commonly used benchmarks demonstrate that EfficientPlace achieves remarkable placement quality within a short timeframe, outperforming recent state-of-the-art approaches.} }
Endnote
%0 Conference Paper %T Reinforcement Learning within Tree Search for Fast Macro Placement %A Zijie Geng %A Jie Wang %A Ziyan Liu %A Siyuan Xu %A Zhentao Tang %A Mingxuan Yuan %A Jianye Hao %A Yongdong Zhang %A Feng Wu %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-geng24b %I PMLR %P 15402--15417 %U https://proceedings.mlr.press/v235/geng24b.html %V 235 %X Macro placement is a crucial step in modern chip design, and reinforcement learning (RL) has recently emerged as a promising technique for improving the placement quality. However, existing RL-based techniques are hindered by their low sample efficiency, requiring numerous online rollouts or substantial offline expert data to achieve bootstrap, which are often impractical in industrial scenarios. To address this challenge, we propose a novel sample-efficient framework, namely EfficientPlace, for fast macro placement. EfficientPlace integrates a global tree search algorithm to strategically direct the optimization process, as well as a RL agent for local policy learning to advance the tree search. Experiments on commonly used benchmarks demonstrate that EfficientPlace achieves remarkable placement quality within a short timeframe, outperforming recent state-of-the-art approaches.
APA
Geng, Z., Wang, J., Liu, Z., Xu, S., Tang, Z., Yuan, M., Hao, J., Zhang, Y. & Wu, F.. (2024). Reinforcement Learning within Tree Search for Fast Macro Placement. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:15402-15417 Available from https://proceedings.mlr.press/v235/geng24b.html.

Related Material