Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks

Amar Gahir, Varshil Patel, Shreyank N Gowda
Proceedings of the Fourth UK AI Conference 2026, PMLR 348:78-87, 2026.

Abstract

Deep neural networks are typically trained by uniformly sampling large datasets across epochs, despite evidence that not all samples contribute equally throughout learning. Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization, but existing methods rely on fixed schedules that do not adapt during training. In this work, we propose Adaptive Data Dropout, a simple framework that dynamically adjusts the subset of training data based on performance feedback. Inspired by self-regulated learning, our approach treats data selection as an adaptive process, increasing or decreasing data exposure in response to changes in training accuracy. We introduce a lightweight stochastic update mechanism that modulates the dropout schedule online, allowing the model to balance exploration and consolidation over time. Experiments on standard image classification benchmarks show that our method reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies. These results highlight adaptive data selection as a promising direction for efficient and robust training. Code will be released.

Cite this Paper

BibTeX
@InProceedings{pmlr-v348-gahir26a, title = {Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks}, author = {Gahir, Amar and Patel, Varshil and Gowda, Shreyank N}, booktitle = {Proceedings of the Fourth UK AI Conference 2026}, pages = {78--87}, year = {2026}, editor = {Benford, Alistair and Büyükateş, Baturalp and Cabrera, Christian and Kiden, Sarah and Salili-James, Arianna and Zakka, Vincent and Zhou, Feng}, volume = {348}, series = {Proceedings of Machine Learning Research}, month = {29--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v348/main/assets/gahir26a/gahir26a.pdf}, url = {https://proceedings.mlr.press/v348/gahir26a.html}, abstract = {Deep neural networks are typically trained by uniformly sampling large datasets across epochs, despite evidence that not all samples contribute equally throughout learning. Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization, but existing methods rely on fixed schedules that do not adapt during training. In this work, we propose Adaptive Data Dropout, a simple framework that dynamically adjusts the subset of training data based on performance feedback. Inspired by self-regulated learning, our approach treats data selection as an adaptive process, increasing or decreasing data exposure in response to changes in training accuracy. We introduce a lightweight stochastic update mechanism that modulates the dropout schedule online, allowing the model to balance exploration and consolidation over time. Experiments on standard image classification benchmarks show that our method reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies. These results highlight adaptive data selection as a promising direction for efficient and robust training. Code will be released.} }
Endnote
%0 Conference Paper %T Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks %A Amar Gahir %A Varshil Patel %A Shreyank N Gowda %B Proceedings of the Fourth UK AI Conference 2026 %C Proceedings of Machine Learning Research %D 2026 %E Alistair Benford %E Baturalp Büyükateş %E Christian Cabrera %E Sarah Kiden %E Arianna Salili-James %E Vincent Zakka %E Feng Zhou %F pmlr-v348-gahir26a %I PMLR %P 78--87 %U https://proceedings.mlr.press/v348/gahir26a.html %V 348 %X Deep neural networks are typically trained by uniformly sampling large datasets across epochs, despite evidence that not all samples contribute equally throughout learning. Recent work shows that progressively reducing the amount of training data can improve efficiency and generalization, but existing methods rely on fixed schedules that do not adapt during training. In this work, we propose Adaptive Data Dropout, a simple framework that dynamically adjusts the subset of training data based on performance feedback. Inspired by self-regulated learning, our approach treats data selection as an adaptive process, increasing or decreasing data exposure in response to changes in training accuracy. We introduce a lightweight stochastic update mechanism that modulates the dropout schedule online, allowing the model to balance exploration and consolidation over time. Experiments on standard image classification benchmarks show that our method reduces effective training steps while maintaining competitive accuracy compared to static data dropout strategies. These results highlight adaptive data selection as a promising direction for efficient and robust training. Code will be released.
APA
Gahir, A., Patel, V. & Gowda, S.N.. (2026). Adaptive Data Dropout: Towards Self-Regulated Learning in Deep Neural Networks. Proceedings of the Fourth UK AI Conference 2026, in Proceedings of Machine Learning Research 348:78-87 Available from https://proceedings.mlr.press/v348/gahir26a.html.

Related Material