Lower-Bounded Proper Losses for Weakly Supervised Classification

Shuhei M Yoshida, Takashi Takenouchi, Masashi Sugiyama
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:12110-12120, 2021.

Abstract

This paper discusses the problem of weakly supervised classification, in which instances are given weak labels that are produced by some label-corruption process. The goal is to derive conditions under which loss functions for weak-label learning are proper and lower-bounded—two essential requirements for the losses used in class-probability estimation. To this end, we derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We use this theorem to characterize proper weak-label losses and find a condition for them to be lower-bounded. From these theoretical findings, we derive a novel regularization scheme called generalized logit squeezing, which makes any proper weak-label loss bounded from below, without losing properness. Furthermore, we experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses. The results highlight the importance of properness and lower-boundedness.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-yoshida21a, title = {Lower-Bounded Proper Losses for Weakly Supervised Classification}, author = {Yoshida, Shuhei M and Takenouchi, Takashi and Sugiyama, Masashi}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {12110--12120}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/yoshida21a/yoshida21a.pdf}, url = {https://proceedings.mlr.press/v139/yoshida21a.html}, abstract = {This paper discusses the problem of weakly supervised classification, in which instances are given weak labels that are produced by some label-corruption process. The goal is to derive conditions under which loss functions for weak-label learning are proper and lower-bounded—two essential requirements for the losses used in class-probability estimation. To this end, we derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We use this theorem to characterize proper weak-label losses and find a condition for them to be lower-bounded. From these theoretical findings, we derive a novel regularization scheme called generalized logit squeezing, which makes any proper weak-label loss bounded from below, without losing properness. Furthermore, we experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses. The results highlight the importance of properness and lower-boundedness.} }
Endnote
%0 Conference Paper %T Lower-Bounded Proper Losses for Weakly Supervised Classification %A Shuhei M Yoshida %A Takashi Takenouchi %A Masashi Sugiyama %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-yoshida21a %I PMLR %P 12110--12120 %U https://proceedings.mlr.press/v139/yoshida21a.html %V 139 %X This paper discusses the problem of weakly supervised classification, in which instances are given weak labels that are produced by some label-corruption process. The goal is to derive conditions under which loss functions for weak-label learning are proper and lower-bounded—two essential requirements for the losses used in class-probability estimation. To this end, we derive a representation theorem for proper losses in supervised learning, which dualizes the Savage representation. We use this theorem to characterize proper weak-label losses and find a condition for them to be lower-bounded. From these theoretical findings, we derive a novel regularization scheme called generalized logit squeezing, which makes any proper weak-label loss bounded from below, without losing properness. Furthermore, we experimentally demonstrate the effectiveness of our proposed approach, as compared to improper or unbounded losses. The results highlight the importance of properness and lower-boundedness.
APA
Yoshida, S.M., Takenouchi, T. & Sugiyama, M.. (2021). Lower-Bounded Proper Losses for Weakly Supervised Classification. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:12110-12120 Available from https://proceedings.mlr.press/v139/yoshida21a.html.

Related Material