Controlling Path Dependence in Gradient Ascent Unlearning through Forget-Set Ordering

Varun Sampath Kumar, Esmaeil S. Nadimi, Vinay Chakravarthi Gogineni
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:3224-3236, 2026.

Abstract

Machine unlearning aims to selectively remove the influence of designated training data from a trained model. Gradient ascent-based unlearning methods are widely used in machine unlearning, but are often observed to exhibit instability and sensitivity to optimization hyperparameters. In this work, we show that the ordering of forget-set samples during ascent materially influences the resulting optimization trajectory, yielding markedly different forgetting–retention trade-offs under identical optimization budgets. We provide a local second-order analysis illustrating how non-commutativity of ascent updates induces path dependence, and we propose a simple ordering strategy that processes forget samples from low to high predictive uncertainty. By deferring high-uncertainty samples, this strategy attenuates early parameter drift and leads to more stable trajectories. Across image classification benchmarks with convolutional and vision transformer architectures, as well as text classification tasks, we observe that this ordering consistently achieves effective forgetting while better preserving retain-set utility compared to random ordering. In the retain-data-free regime, this trajectory-aware modification substantially strengthens vanilla gradient ascent, improving its position on the forgetting–retention Pareto frontier relative to naive ascent and several retain-data-free unlearning baselines. In general, our results identify sample ordering as a practically meaningful and previously under explored degree of freedom in machine unlearning, highlighting the importance of trajectory-aware design and evaluation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-kumar26a, title = {Controlling Path Dependence in Gradient Ascent Unlearning through Forget-Set Ordering}, author = {Kumar, Varun Sampath and Nadimi, Esmaeil S. and Gogineni, Vinay Chakravarthi}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {3224--3236}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/kumar26a/kumar26a.pdf}, url = {https://proceedings.mlr.press/v337/kumar26a.html}, abstract = {Machine unlearning aims to selectively remove the influence of designated training data from a trained model. Gradient ascent-based unlearning methods are widely used in machine unlearning, but are often observed to exhibit instability and sensitivity to optimization hyperparameters. In this work, we show that the ordering of forget-set samples during ascent materially influences the resulting optimization trajectory, yielding markedly different forgetting–retention trade-offs under identical optimization budgets. We provide a local second-order analysis illustrating how non-commutativity of ascent updates induces path dependence, and we propose a simple ordering strategy that processes forget samples from low to high predictive uncertainty. By deferring high-uncertainty samples, this strategy attenuates early parameter drift and leads to more stable trajectories. Across image classification benchmarks with convolutional and vision transformer architectures, as well as text classification tasks, we observe that this ordering consistently achieves effective forgetting while better preserving retain-set utility compared to random ordering. In the retain-data-free regime, this trajectory-aware modification substantially strengthens vanilla gradient ascent, improving its position on the forgetting–retention Pareto frontier relative to naive ascent and several retain-data-free unlearning baselines. In general, our results identify sample ordering as a practically meaningful and previously under explored degree of freedom in machine unlearning, highlighting the importance of trajectory-aware design and evaluation.} }
Endnote
%0 Conference Paper %T Controlling Path Dependence in Gradient Ascent Unlearning through Forget-Set Ordering %A Varun Sampath Kumar %A Esmaeil S. Nadimi %A Vinay Chakravarthi Gogineni %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-kumar26a %I PMLR %P 3224--3236 %U https://proceedings.mlr.press/v337/kumar26a.html %V 337 %X Machine unlearning aims to selectively remove the influence of designated training data from a trained model. Gradient ascent-based unlearning methods are widely used in machine unlearning, but are often observed to exhibit instability and sensitivity to optimization hyperparameters. In this work, we show that the ordering of forget-set samples during ascent materially influences the resulting optimization trajectory, yielding markedly different forgetting–retention trade-offs under identical optimization budgets. We provide a local second-order analysis illustrating how non-commutativity of ascent updates induces path dependence, and we propose a simple ordering strategy that processes forget samples from low to high predictive uncertainty. By deferring high-uncertainty samples, this strategy attenuates early parameter drift and leads to more stable trajectories. Across image classification benchmarks with convolutional and vision transformer architectures, as well as text classification tasks, we observe that this ordering consistently achieves effective forgetting while better preserving retain-set utility compared to random ordering. In the retain-data-free regime, this trajectory-aware modification substantially strengthens vanilla gradient ascent, improving its position on the forgetting–retention Pareto frontier relative to naive ascent and several retain-data-free unlearning baselines. In general, our results identify sample ordering as a practically meaningful and previously under explored degree of freedom in machine unlearning, highlighting the importance of trajectory-aware design and evaluation.
APA
Kumar, V.S., Nadimi, E.S. & Gogineni, V.C.. (2026). Controlling Path Dependence in Gradient Ascent Unlearning through Forget-Set Ordering. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:3224-3236 Available from https://proceedings.mlr.press/v337/kumar26a.html.

Related Material