High Order Regularization for Semi-Supervised Learning of Structured Output Problems

Yujia Li, Rich Zemel
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1368-1376, 2014.

Abstract

Semi-supervised learning, which uses unlabeled data to help learn a discriminative model, is especially important for structured output problems, as considerably more effort is needed to label its multidimensional outputs versus standard single output problems. We propose a new max-margin framework for semi-supervised structured output learning, that allows the use of powerful discrete optimization algorithms and high order regularizers defined directly on model predictions for the unlabeled examples. We show that our framework is closely related to Posterior Regularization, and the two frameworks optimize special cases of the same objective. The new framework is instantiated on two image segmentation tasks, using both a graph regularizer and a cardinality regularizer. Experiments also demonstrate that this framework can utilize unlabeled data from a different source than the labeled data to significantly improve performance while saving labeling effort.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-lif14, title = {High Order Regularization for Semi-Supervised Learning of Structured Output Problems}, author = {Li, Yujia and Zemel, Rich}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1368--1376}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/lif14.pdf}, url = {https://proceedings.mlr.press/v32/lif14.html}, abstract = {Semi-supervised learning, which uses unlabeled data to help learn a discriminative model, is especially important for structured output problems, as considerably more effort is needed to label its multidimensional outputs versus standard single output problems. We propose a new max-margin framework for semi-supervised structured output learning, that allows the use of powerful discrete optimization algorithms and high order regularizers defined directly on model predictions for the unlabeled examples. We show that our framework is closely related to Posterior Regularization, and the two frameworks optimize special cases of the same objective. The new framework is instantiated on two image segmentation tasks, using both a graph regularizer and a cardinality regularizer. Experiments also demonstrate that this framework can utilize unlabeled data from a different source than the labeled data to significantly improve performance while saving labeling effort.} }
Endnote
%0 Conference Paper %T High Order Regularization for Semi-Supervised Learning of Structured Output Problems %A Yujia Li %A Rich Zemel %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-lif14 %I PMLR %P 1368--1376 %U https://proceedings.mlr.press/v32/lif14.html %V 32 %N 2 %X Semi-supervised learning, which uses unlabeled data to help learn a discriminative model, is especially important for structured output problems, as considerably more effort is needed to label its multidimensional outputs versus standard single output problems. We propose a new max-margin framework for semi-supervised structured output learning, that allows the use of powerful discrete optimization algorithms and high order regularizers defined directly on model predictions for the unlabeled examples. We show that our framework is closely related to Posterior Regularization, and the two frameworks optimize special cases of the same objective. The new framework is instantiated on two image segmentation tasks, using both a graph regularizer and a cardinality regularizer. Experiments also demonstrate that this framework can utilize unlabeled data from a different source than the labeled data to significantly improve performance while saving labeling effort.
RIS
TY - CPAPER TI - High Order Regularization for Semi-Supervised Learning of Structured Output Problems AU - Yujia Li AU - Rich Zemel BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-lif14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1368 EP - 1376 L1 - http://proceedings.mlr.press/v32/lif14.pdf UR - https://proceedings.mlr.press/v32/lif14.html AB - Semi-supervised learning, which uses unlabeled data to help learn a discriminative model, is especially important for structured output problems, as considerably more effort is needed to label its multidimensional outputs versus standard single output problems. We propose a new max-margin framework for semi-supervised structured output learning, that allows the use of powerful discrete optimization algorithms and high order regularizers defined directly on model predictions for the unlabeled examples. We show that our framework is closely related to Posterior Regularization, and the two frameworks optimize special cases of the same objective. The new framework is instantiated on two image segmentation tasks, using both a graph regularizer and a cardinality regularizer. Experiments also demonstrate that this framework can utilize unlabeled data from a different source than the labeled data to significantly improve performance while saving labeling effort. ER -
APA
Li, Y. & Zemel, R.. (2014). High Order Regularization for Semi-Supervised Learning of Structured Output Problems. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1368-1376 Available from https://proceedings.mlr.press/v32/lif14.html.

Related Material