Tool Unlearning for Tool-Augmented LLMs

Jiali Cheng, Hadi Amiri
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:9947-9963, 2025.

Abstract

Tool-augmented large language models (LLMs) may need to forget learned tools due to security concerns, privacy restrictions, or deprecated tools. However, “tool unlearning” has not been investigated in machine unlearning literature. We introduce this novel task, which requires addressing distinct challenges compared to traditional unlearning: knowledge removal rather than forgetting individual samples, the high cost of optimizing LLMs, and the need for principled evaluation metrics. To bridge these gaps, we propose ToolDelete , the first approach for unlearning tools from tool-augmented LLMs which implements three properties for effective tool unlearning, and a new membership inference attack (MIA) model for evaluation. Experiments on three tool learning datasets and tool-augmented LLMs show that ToolDelete effectively unlearns both randomly selected and category-specific tools, while preserving the LLM’s knowledge on non-deleted tools and maintaining performance on general tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-cheng25a, title = {Tool Unlearning for Tool-Augmented {LLM}s}, author = {Cheng, Jiali and Amiri, Hadi}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {9947--9963}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/cheng25a/cheng25a.pdf}, url = {https://proceedings.mlr.press/v267/cheng25a.html}, abstract = {Tool-augmented large language models (LLMs) may need to forget learned tools due to security concerns, privacy restrictions, or deprecated tools. However, “tool unlearning” has not been investigated in machine unlearning literature. We introduce this novel task, which requires addressing distinct challenges compared to traditional unlearning: knowledge removal rather than forgetting individual samples, the high cost of optimizing LLMs, and the need for principled evaluation metrics. To bridge these gaps, we propose ToolDelete , the first approach for unlearning tools from tool-augmented LLMs which implements three properties for effective tool unlearning, and a new membership inference attack (MIA) model for evaluation. Experiments on three tool learning datasets and tool-augmented LLMs show that ToolDelete effectively unlearns both randomly selected and category-specific tools, while preserving the LLM’s knowledge on non-deleted tools and maintaining performance on general tasks.} }
Endnote
%0 Conference Paper %T Tool Unlearning for Tool-Augmented LLMs %A Jiali Cheng %A Hadi Amiri %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-cheng25a %I PMLR %P 9947--9963 %U https://proceedings.mlr.press/v267/cheng25a.html %V 267 %X Tool-augmented large language models (LLMs) may need to forget learned tools due to security concerns, privacy restrictions, or deprecated tools. However, “tool unlearning” has not been investigated in machine unlearning literature. We introduce this novel task, which requires addressing distinct challenges compared to traditional unlearning: knowledge removal rather than forgetting individual samples, the high cost of optimizing LLMs, and the need for principled evaluation metrics. To bridge these gaps, we propose ToolDelete , the first approach for unlearning tools from tool-augmented LLMs which implements three properties for effective tool unlearning, and a new membership inference attack (MIA) model for evaluation. Experiments on three tool learning datasets and tool-augmented LLMs show that ToolDelete effectively unlearns both randomly selected and category-specific tools, while preserving the LLM’s knowledge on non-deleted tools and maintaining performance on general tasks.
APA
Cheng, J. & Amiri, H.. (2025). Tool Unlearning for Tool-Augmented LLMs. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:9947-9963 Available from https://proceedings.mlr.press/v267/cheng25a.html.

Related Material