One-Class SVM-guided Negative Sampling for Enhanced Contrastive Learning

Dhruv Jain, Tsiry Mayet, Romain HÉRAULT, Romain MODZELEWSKI
Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL), PMLR 265:110-119, 2025.

Abstract

Recent studies on contrastive learning have emphasized carefully sampling and mixing negative samples. This study introduces a novel and improved approach for generating synthetic negatives. We propose a new method using One-Class Support Vector Machine (OCSVM) to guide in the selection process before mixing named as **Mixing OCSVM negatives (MiOC)**. Our results show that our approach creates more meaningful embeddings, which lead to better classification performance. We implement our method using publicly available datasets (Imagenet100, Cifar10, Cifar100, Cinic10, and STL10). We observed that MiOC exhibit favorable performance compared to state-of-the-art methods across these datasets. By presenting a novel approach, this study emphasizes the exploration of alternative mixing techniques that expand the sampling space beyond the conventional confines of hard negatives produced by the ranking of the dot product.

Cite this Paper

BibTeX
@InProceedings{pmlr-v265-jain25a, title = {One-Class {SVM}-guided Negative Sampling for Enhanced Contrastive Learning}, author = {Jain, Dhruv and Mayet, Tsiry and H{\'E}RAULT, Romain and MODZELEWSKI, Romain}, booktitle = {Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL)}, pages = {110--119}, year = {2025}, editor = {Lutchyn, Tetiana and Ramírez Rivera, Adín and Ricaud, Benjamin}, volume = {265}, series = {Proceedings of Machine Learning Research}, month = {07--09 Jan}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v265/main/assets/jain25a/jain25a.pdf}, url = {https://proceedings.mlr.press/v265/jain25a.html}, abstract = {Recent studies on contrastive learning have emphasized carefully sampling and mixing negative samples. This study introduces a novel and improved approach for generating synthetic negatives. We propose a new method using One-Class Support Vector Machine (OCSVM) to guide in the selection process before mixing named as **Mixing OCSVM negatives (MiOC)**. Our results show that our approach creates more meaningful embeddings, which lead to better classification performance. We implement our method using publicly available datasets (Imagenet100, Cifar10, Cifar100, Cinic10, and STL10). We observed that MiOC exhibit favorable performance compared to state-of-the-art methods across these datasets. By presenting a novel approach, this study emphasizes the exploration of alternative mixing techniques that expand the sampling space beyond the conventional confines of hard negatives produced by the ranking of the dot product.} }
Endnote
%0 Conference Paper %T One-Class SVM-guided Negative Sampling for Enhanced Contrastive Learning %A Dhruv Jain %A Tsiry Mayet %A Romain HÉRAULT %A Romain MODZELEWSKI %B Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL) %C Proceedings of Machine Learning Research %D 2025 %E Tetiana Lutchyn %E Adín Ramírez Rivera %E Benjamin Ricaud %F pmlr-v265-jain25a %I PMLR %P 110--119 %U https://proceedings.mlr.press/v265/jain25a.html %V 265 %X Recent studies on contrastive learning have emphasized carefully sampling and mixing negative samples. This study introduces a novel and improved approach for generating synthetic negatives. We propose a new method using One-Class Support Vector Machine (OCSVM) to guide in the selection process before mixing named as **Mixing OCSVM negatives (MiOC)**. Our results show that our approach creates more meaningful embeddings, which lead to better classification performance. We implement our method using publicly available datasets (Imagenet100, Cifar10, Cifar100, Cinic10, and STL10). We observed that MiOC exhibit favorable performance compared to state-of-the-art methods across these datasets. By presenting a novel approach, this study emphasizes the exploration of alternative mixing techniques that expand the sampling space beyond the conventional confines of hard negatives produced by the ranking of the dot product.
APA
Jain, D., Mayet, T., HÉRAULT, R. & MODZELEWSKI, R.. (2025). One-Class SVM-guided Negative Sampling for Enhanced Contrastive Learning. Proceedings of the 6th Northern Lights Deep Learning Conference (NLDL), in Proceedings of Machine Learning Research 265:110-119 Available from https://proceedings.mlr.press/v265/jain25a.html.

Related Material