$f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD

Fajie YUAN, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, CHUA Tat-Seng, Joemon Jose
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:197-206, 2018.

Abstract

Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural language processing (NLP), computer vision (CV), information retrieval (IR). Considering the numerous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to increase computational efficiency. However, sampling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction performance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sampling. By leveraging sparsity in PU data, we accelerate fBGD by several magnitudes, making its time complexity the same level as the NS-based stochastic gradient descent method. Meanwhile, we observe that the standard batch gradient method suffers from gradient instability issues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an intuitive solution arises naturally. To verify its efficacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based models on all tasks with comparable efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-yuan18a, title = {$f_{BGD}$: Learning Embeddings From Positive Unlabeled Data with {BGD}}, author = {YUAN, Fajie and Xin, Xin and He, Xiangnan and Guo, Guibing and Zhang, Weinan and Tat-Seng, CHUA and Jose, Joemon}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {197--206}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/yuan18a/yuan18a.pdf}, url = {https://proceedings.mlr.press/r16/yuan18a.html}, abstract = {Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural language processing (NLP), computer vision (CV), information retrieval (IR). Considering the numerous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to increase computational efficiency. However, sampling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction performance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sampling. By leveraging sparsity in PU data, we accelerate fBGD by several magnitudes, making its time complexity the same level as the NS-based stochastic gradient descent method. Meanwhile, we observe that the standard batch gradient method suffers from gradient instability issues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an intuitive solution arises naturally. To verify its efficacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based models on all tasks with comparable efficiency.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T $f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD %A Fajie YUAN %A Xin Xin %A Xiangnan He %A Guibing Guo %A Weinan Zhang %A CHUA Tat-Seng %A Joemon Jose %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-yuan18a %I PMLR %P 197--206 %U https://proceedings.mlr.press/r16/yuan18a.html %V R16 %X Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural language processing (NLP), computer vision (CV), information retrieval (IR). Considering the numerous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to increase computational efficiency. However, sampling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction performance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sampling. By leveraging sparsity in PU data, we accelerate fBGD by several magnitudes, making its time complexity the same level as the NS-based stochastic gradient descent method. Meanwhile, we observe that the standard batch gradient method suffers from gradient instability issues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an intuitive solution arises naturally. To verify its efficacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based models on all tasks with comparable efficiency. %Z Reissued by PMLR on 04 October 2026.
APA
YUAN, F., Xin, X., He, X., Guo, G., Zhang, W., Tat-Seng, C. & Jose, J.. (2018). $f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:197-206 Available from https://proceedings.mlr.press/r16/yuan18a.html. Reissued by PMLR on 04 October 2026.

Related Material