Deterministic Annealing for Semi-Supervised Structured Output Learning

Paramveer Dhillon, Sathiya Keerthi, Kedar Bellare, Olivier Chapelle, Sundararajan Sellamanickam
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:299-307, 2012.

Abstract

In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-dhillon12, title = {Deterministic Annealing for Semi-Supervised Structured Output Learning}, author = {Dhillon, Paramveer and Keerthi, Sathiya and Bellare, Kedar and Chapelle, Olivier and Sellamanickam, Sundararajan}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {299--307}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/dhillon12/dhillon12.pdf}, url = {https://proceedings.mlr.press/v22/dhillon12.html}, abstract = {In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).} }
Endnote
%0 Conference Paper %T Deterministic Annealing for Semi-Supervised Structured Output Learning %A Paramveer Dhillon %A Sathiya Keerthi %A Kedar Bellare %A Olivier Chapelle %A Sundararajan Sellamanickam %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-dhillon12 %I PMLR %P 299--307 %U https://proceedings.mlr.press/v22/dhillon12.html %V 22 %X In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).
RIS
TY - CPAPER TI - Deterministic Annealing for Semi-Supervised Structured Output Learning AU - Paramveer Dhillon AU - Sathiya Keerthi AU - Kedar Bellare AU - Olivier Chapelle AU - Sundararajan Sellamanickam BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-dhillon12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 299 EP - 307 L1 - http://proceedings.mlr.press/v22/dhillon12/dhillon12.pdf UR - https://proceedings.mlr.press/v22/dhillon12.html AB - In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR). ER -
APA
Dhillon, P., Keerthi, S., Bellare, K., Chapelle, O. & Sellamanickam, S.. (2012). Deterministic Annealing for Semi-Supervised Structured Output Learning. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:299-307 Available from https://proceedings.mlr.press/v22/dhillon12.html.

Related Material