In value-based deep reinforcement learning, a pruned network is a good network

Johan Samir Obando Ceron, Aaron Courville, Pablo Samuel Castro
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:38495-38519, 2024.

Abstract

Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of sparse training techniques and demonstrate that gradual magnitude pruning enables value-based agents to maximize parameter effectiveness. This results in networks that yield dramatic performance improvements over traditional networks, using only a small fraction of the full network parameters. Our code is publicly available, see Appendix A for details.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-obando-ceron24a, title = {In value-based deep reinforcement learning, a pruned network is a good network}, author = {Obando Ceron, Johan Samir and Courville, Aaron and Castro, Pablo Samuel}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {38495--38519}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/obando-ceron24a/obando-ceron24a.pdf}, url = {https://proceedings.mlr.press/v235/obando-ceron24a.html}, abstract = {Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of sparse training techniques and demonstrate that gradual magnitude pruning enables value-based agents to maximize parameter effectiveness. This results in networks that yield dramatic performance improvements over traditional networks, using only a small fraction of the full network parameters. Our code is publicly available, see Appendix A for details.} }
Endnote
%0 Conference Paper %T In value-based deep reinforcement learning, a pruned network is a good network %A Johan Samir Obando Ceron %A Aaron Courville %A Pablo Samuel Castro %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-obando-ceron24a %I PMLR %P 38495--38519 %U https://proceedings.mlr.press/v235/obando-ceron24a.html %V 235 %X Recent work has shown that deep reinforcement learning agents have difficulty in effectively using their network parameters. We leverage prior insights into the advantages of sparse training techniques and demonstrate that gradual magnitude pruning enables value-based agents to maximize parameter effectiveness. This results in networks that yield dramatic performance improvements over traditional networks, using only a small fraction of the full network parameters. Our code is publicly available, see Appendix A for details.
APA
Obando Ceron, J.S., Courville, A. & Castro, P.S.. (2024). In value-based deep reinforcement learning, a pruned network is a good network. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:38495-38519 Available from https://proceedings.mlr.press/v235/obando-ceron24a.html.

Related Material