$f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD

Fajie YUAN, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, CHUA Tat-Seng, Joemon Jose
Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, PMLR R16:197-206, 2018.

Abstract

Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural lan- guage processing (NLP), computer vision (CV), information retrieval (IR). Considering the nu- merous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to in- crease computational efficiency. However, sam- pling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction perfor- mance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sam- pling. By leveraging sparsity in PU data, we ac- celerate fBGD by several magnitudes, making its time complexity the same level as the NS- based stochastic gradient descent method. Mean- while, we observe that the standard batch gradi- ent method suffers from gradient instability is- sues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an in- tuitive solution arises naturally. To verify its effi- cacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based mod- els on all tasks with comparable efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR16-yuan18a, title = {$f_{BGD}$: Learning Embeddings From Positive Unlabeled Data with {BGD}}, author = {YUAN, Fajie and Xin, Xin and He, Xiangnan and Guo, Guibing and Zhang, Weinan and Tat-Seng, CHUA and Jose, Joemon}, booktitle = {Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence}, pages = {197--206}, year = {2018}, editor = {Globerson, Amir and Silva, Ricardo}, volume = {R16}, series = {Proceedings of Machine Learning Research}, month = {06--10 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r16/main/assets/yuan18a/yuan18a.pdf}, url = {https://proceedings.mlr.press/r16/yuan18a.html}, abstract = {Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural lan- guage processing (NLP), computer vision (CV), information retrieval (IR). Considering the nu- merous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to in- crease computational efficiency. However, sam- pling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction perfor- mance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sam- pling. By leveraging sparsity in PU data, we ac- celerate fBGD by several magnitudes, making its time complexity the same level as the NS- based stochastic gradient descent method. Mean- while, we observe that the standard batch gradi- ent method suffers from gradient instability is- sues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an in- tuitive solution arises naturally. To verify its effi- cacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based mod- els on all tasks with comparable efficiency.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T $f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD %A Fajie YUAN %A Xin Xin %A Xiangnan He %A Guibing Guo %A Weinan Zhang %A CHUA Tat-Seng %A Joemon Jose %B Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2018 %E Amir Globerson %E Ricardo Silva %F pmlr-vR16-yuan18a %I PMLR %P 197--206 %U https://proceedings.mlr.press/r16/yuan18a.html %V R16 %X Learning sparse features from only positive and unlabeled (PU) data is a fundamental task for problems of several domains, such as natural lan- guage processing (NLP), computer vision (CV), information retrieval (IR). Considering the nu- merous amount of unlabeled data, most prevalent methods rely on negative sampling (NS) to in- crease computational efficiency. However, sam- pling a fraction of unlabeled data as negative for training may ignore other important examples, and thus lead to non-optimal prediction perfor- mance. To address this, we present a fast and generic batch gradient descent optimizer (fBGD) to learn from all training examples without sam- pling. By leveraging sparsity in PU data, we ac- celerate fBGD by several magnitudes, making its time complexity the same level as the NS- based stochastic gradient descent method. Mean- while, we observe that the standard batch gradi- ent method suffers from gradient instability is- sues due to the sparsity property. Driven by a theoretical analysis for this potential cause, an in- tuitive solution arises naturally. To verify its effi- cacy, we perform experiments on multiple tasks with PU data across domains, and show that fBGD consistently outperforms NS-based mod- els on all tasks with comparable efficiency. %Z Reissued by PMLR on 04 October 2026.
APA
YUAN, F., Xin, X., He, X., Guo, G., Zhang, W., Tat-Seng, C. & Jose, J.. (2018). $f_BGD$: Learning Embeddings From Positive Unlabeled Data with BGD. Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R16:197-206 Available from https://proceedings.mlr.press/r16/yuan18a.html. Reissued by PMLR on 04 October 2026.

Related Material