Alternating Projections for Learning with Expectation Constraints

Kedar Bellare, Gregory Druck, Andrew McCallum
Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, PMLR R7:35-42, 2009.

Abstract

We present an objective function for learning with unlabeled data that utilizes auxiliary expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than generalized expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to generalized expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR7-bellare09a, title = {Alternating Projections for Learning with Expectation Constraints}, author = {Bellare, Kedar and Druck, Gregory and McCallum, Andrew}, booktitle = {Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence}, pages = {35--42}, year = {2009}, editor = {Bilmes, Jeff and Ng, Andrew Y.}, volume = {R7}, series = {Proceedings of Machine Learning Research}, month = {18--21 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r7/main/assets/bellare09a/bellare09a.pdf}, url = {https://proceedings.mlr.press/r7/bellare09a.html}, abstract = {We present an objective function for learning with unlabeled data that utilizes auxiliary expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than generalized expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to generalized expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Alternating Projections for Learning with Expectation Constraints %A Kedar Bellare %A Gregory Druck %A Andrew McCallum %B Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2009 %E Jeff Bilmes %E Andrew Y. Ng %F pmlr-vR7-bellare09a %I PMLR %P 35--42 %U https://proceedings.mlr.press/r7/bellare09a.html %V R7 %X We present an objective function for learning with unlabeled data that utilizes auxiliary expectation constraints. We optimize this objective function using a procedure that alternates between information and moment projections. Our method provides an alternate interpretation of the posterior regularization framework (Graca et al., 2008), maintains uncertainty during optimization unlike constraint-driven learning (Chang et al., 2007), and is more efficient than generalized expectation criteria (Mann & McCallum, 2008). Applications of this framework include minimally supervised learning, semisupervised learning, and learning with constraints that are more expressive than the underlying model. In experiments, we demonstrate comparable accuracy to generalized expectation criteria for minimally supervised learning, and use expressive structural constraints to guide semi-supervised learning, providing a 3%-6% improvement over stateof-the-art constraint-driven learning. %Z Reissued by PMLR on 04 October 2026.
APA
Bellare, K., Druck, G. & McCallum, A.. (2009). Alternating Projections for Learning with Expectation Constraints. Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R7:35-42 Available from https://proceedings.mlr.press/r7/bellare09a.html. Reissued by PMLR on 04 October 2026.

Related Material