Hidden-Unit Conditional Random Fields

Laurens van der Maaten, Max Welling, Lawrence Saul
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:479-488, 2011.

Abstract

The paper explores a generalization of conditional random fields (CRFs) in which binary stochastic hidden units appear between the data and the labels. Hidden-unit CRFs are potentially more powerful than standard CRFs because they can represent nonlinear dependencies at each frame. The hidden units in these models also learn to discover latent distributed structure in the data that improves classification. We derive efficient algorithms for inference and learning in these models by observing that the hidden units are conditionally independent given the data and the labels. Finally, we show that hidden-unit CRFs perform well in experiments on a range of tasks, including optical character recognition, text classification, protein structure prediction, and part-of-speech tagging.

Cite this Paper

BibTeX
@InProceedings{pmlr-v15-maaten11b, title = {Hidden-Unit Conditional Random Fields}, author = {van der Maaten, Laurens and Welling, Max and Saul, Lawrence}, booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, pages = {479--488}, year = {2011}, editor = {Gordon, Geoffrey and Dunson, David and Dudík, Miroslav}, volume = {15}, series = {Proceedings of Machine Learning Research}, address = {Fort Lauderdale, FL, USA}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v15/maaten11b/maaten11b.pdf}, url = {https://proceedings.mlr.press/v15/maaten11b.html}, abstract = {The paper explores a generalization of conditional random fields (CRFs) in which binary stochastic hidden units appear between the data and the labels. Hidden-unit CRFs are potentially more powerful than standard CRFs because they can represent nonlinear dependencies at each frame. The hidden units in these models also learn to discover latent distributed structure in the data that improves classification. We derive efficient algorithms for inference and learning in these models by observing that the hidden units are conditionally independent given the data and the labels. Finally, we show that hidden-unit CRFs perform well in experiments on a range of tasks, including optical character recognition, text classification, protein structure prediction, and part-of-speech tagging.} }
Endnote
%0 Conference Paper %T Hidden-Unit Conditional Random Fields %A Laurens van der Maaten %A Max Welling %A Lawrence Saul %B Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2011 %E Geoffrey Gordon %E David Dunson %E Miroslav Dudík %F pmlr-v15-maaten11b %I PMLR %P 479--488 %U https://proceedings.mlr.press/v15/maaten11b.html %V 15 %X The paper explores a generalization of conditional random fields (CRFs) in which binary stochastic hidden units appear between the data and the labels. Hidden-unit CRFs are potentially more powerful than standard CRFs because they can represent nonlinear dependencies at each frame. The hidden units in these models also learn to discover latent distributed structure in the data that improves classification. We derive efficient algorithms for inference and learning in these models by observing that the hidden units are conditionally independent given the data and the labels. Finally, we show that hidden-unit CRFs perform well in experiments on a range of tasks, including optical character recognition, text classification, protein structure prediction, and part-of-speech tagging.
RIS
TY - CPAPER TI - Hidden-Unit Conditional Random Fields AU - Laurens van der Maaten AU - Max Welling AU - Lawrence Saul BT - Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics DA - 2011/06/14 ED - Geoffrey Gordon ED - David Dunson ED - Miroslav Dudík ID - pmlr-v15-maaten11b PB - PMLR DP - Proceedings of Machine Learning Research VL - 15 SP - 479 EP - 488 L1 - http://proceedings.mlr.press/v15/maaten11b/maaten11b.pdf UR - https://proceedings.mlr.press/v15/maaten11b.html AB - The paper explores a generalization of conditional random fields (CRFs) in which binary stochastic hidden units appear between the data and the labels. Hidden-unit CRFs are potentially more powerful than standard CRFs because they can represent nonlinear dependencies at each frame. The hidden units in these models also learn to discover latent distributed structure in the data that improves classification. We derive efficient algorithms for inference and learning in these models by observing that the hidden units are conditionally independent given the data and the labels. Finally, we show that hidden-unit CRFs perform well in experiments on a range of tasks, including optical character recognition, text classification, protein structure prediction, and part-of-speech tagging. ER -
APA
van der Maaten, L., Welling, M. & Saul, L.. (2011). Hidden-Unit Conditional Random Fields. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 15:479-488 Available from https://proceedings.mlr.press/v15/maaten11b.html.

Related Material