[edit]
Near-Optimal Algorithms for Private Online Optimization in the Realizable Regime
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:1107-1120, 2023.
Abstract
We consider online learning problems in the realizable setting, where there is a zero-loss solution, and propose new Differentially Private (DP) algorithms that obtain near-optimal regret bounds. For the problem of online prediction from experts, we design new algorithms that obtain near-optimal regret O(ε−1poly(logd)) where d is the number of experts. This significantly improves over the best existing regret bounds for the DP non-realizable setting which are O(ε−1min. We also develop an adaptive algorithm for the small-loss setting with regret (L^\star+ \varepsilon^{-1}) \cdot O(\mathsf{poly}(\log{d})) where L^\star is the total loss of the best expert. Additionally, we consider DP online convex optimization in the realizable setting and propose an algorithm with near-optimal regret O \big(\varepsilon^{-1} \mathsf{poly}(d) \big), as well as an algorithm for the smooth case with regret O \big( (\sqrt{Td}/\varepsilon)^{2/3} \big), both significantly improving over existing bounds in the non-realizable regime.