Weighted distance nearest neighbor condensing

Lee-Ad Gottlieb, Timor Sharabi, Roi Weiss
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:16153-16166, 2024.

Abstract

The problem of nearest neighbor condensing has enjoyed a long history of study, both in its theoretical and practical aspects. In this paper, we introduce the problem of weighted distance nearest neighbor condensing, where one assigns weights to each point of the condensed set, and then new points are labeled based on their weighted distance nearest neighbor in the condensed set. We study the theoretical properties of this new model, and show that it can produce dramatically better condensing than the standard nearest neighbor rule, yet is characterized by generalization bounds almost identical to the latter. We then suggest a condensing heuristic for our new problem. We demonstrate Bayes consistency for this heuristic, and also show promising empirical results.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-gottlieb24a, title = {Weighted distance nearest neighbor condensing}, author = {Gottlieb, Lee-Ad and Sharabi, Timor and Weiss, Roi}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {16153--16166}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/gottlieb24a/gottlieb24a.pdf}, url = {https://proceedings.mlr.press/v235/gottlieb24a.html}, abstract = {The problem of nearest neighbor condensing has enjoyed a long history of study, both in its theoretical and practical aspects. In this paper, we introduce the problem of weighted distance nearest neighbor condensing, where one assigns weights to each point of the condensed set, and then new points are labeled based on their weighted distance nearest neighbor in the condensed set. We study the theoretical properties of this new model, and show that it can produce dramatically better condensing than the standard nearest neighbor rule, yet is characterized by generalization bounds almost identical to the latter. We then suggest a condensing heuristic for our new problem. We demonstrate Bayes consistency for this heuristic, and also show promising empirical results.} }
Endnote
%0 Conference Paper %T Weighted distance nearest neighbor condensing %A Lee-Ad Gottlieb %A Timor Sharabi %A Roi Weiss %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-gottlieb24a %I PMLR %P 16153--16166 %U https://proceedings.mlr.press/v235/gottlieb24a.html %V 235 %X The problem of nearest neighbor condensing has enjoyed a long history of study, both in its theoretical and practical aspects. In this paper, we introduce the problem of weighted distance nearest neighbor condensing, where one assigns weights to each point of the condensed set, and then new points are labeled based on their weighted distance nearest neighbor in the condensed set. We study the theoretical properties of this new model, and show that it can produce dramatically better condensing than the standard nearest neighbor rule, yet is characterized by generalization bounds almost identical to the latter. We then suggest a condensing heuristic for our new problem. We demonstrate Bayes consistency for this heuristic, and also show promising empirical results.
APA
Gottlieb, L., Sharabi, T. & Weiss, R.. (2024). Weighted distance nearest neighbor condensing. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:16153-16166 Available from https://proceedings.mlr.press/v235/gottlieb24a.html.

Related Material