Deterministic Annealing for Semi-Supervised Structured Output Learning

Paramveer Dhillon, Sathiya Keerthi, Kedar Bellare, Olivier Chapelle, Sundararajan Sellamanickam
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:299-307, 2012.

Abstract

In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-dhillon12, title = {Deterministic Annealing for Semi-Supervised Structured Output Learning}, author = {Paramveer Dhillon and Sathiya Keerthi and Kedar Bellare and Olivier Chapelle and Sundararajan Sellamanickam}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {299--307}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/dhillon12/dhillon12.pdf}, url = {http://proceedings.mlr.press/v22/dhillon12.html}, abstract = {In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).} }
Endnote
%0 Conference Paper %T Deterministic Annealing for Semi-Supervised Structured Output Learning %A Paramveer Dhillon %A Sathiya Keerthi %A Kedar Bellare %A Olivier Chapelle %A Sundararajan Sellamanickam %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-dhillon12 %I PMLR %J Proceedings of Machine Learning Research %P 299--307 %U http://proceedings.mlr.press %V 22 %W PMLR %X In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR).
RIS
TY - CPAPER TI - Deterministic Annealing for Semi-Supervised Structured Output Learning AU - Paramveer Dhillon AU - Sathiya Keerthi AU - Kedar Bellare AU - Olivier Chapelle AU - Sundararajan Sellamanickam BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-dhillon12 PB - PMLR SP - 299 DP - PMLR EP - 307 L1 - http://proceedings.mlr.press/v22/dhillon12/dhillon12.pdf UR - http://proceedings.mlr.press/v22/dhillon12.html AB - In this paper we propose a new approach for semi-supervised structured output learning. Our approach uses relaxed labeling on unlabeled data to deal with the combinatorial nature of the label space and further uses domain constraints to guide the learning. Since the overall objective is non-convex, we alternate between the optimization of the model parameters and the label distribution of unlabeled data. The alternating optimization coupled with deterministic annealing helps us achieve better local optima and as a result our approach leads to better constraint satisfaction during inference. Experimental results on sequence labeling benchmarks show superior performance of our approach compared to Constraint Driven Learning (CoDL) and Posterior Regularization (PR). ER -
APA
Dhillon, P., Keerthi, S., Bellare, K., Chapelle, O. & Sellamanickam, S.. (2012). Deterministic Annealing for Semi-Supervised Structured Output Learning. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:299-307

Related Material