Asymptotic Theory for Linear-Chain Conditional Random Fields

Mathieu Sinn, Pascal Poupart
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:679-687, 2011.

Abstract

In this theoretical paper we develop an asymptotic theory for Linear-Chain Conditional Random Fields (L-CRFs) and apply it to derive conditions under which the Maximum Likelihood Estimates (MLEs) of the model weights are strongly consistent. We first define L-CRFs for infinite sequences and analyze some of their basic properties. Then we establish conditions under which ergodicity of the observations implies ergodicity of the joint sequence of observations and labels. This result is the key ingredient to derive conditions for strong consistency of the MLEs. Interesting findings are that the consistency crucially depends on the limit behavior of the Hessian of the likelihood function and that, asymptotically, the state feature functions do not matter.

Cite this Paper

BibTeX
@InProceedings{pmlr-v15-sinn11a, title = {Asymptotic Theory for Linear-Chain Conditional Random Fields}, author = {Sinn, Mathieu and Poupart, Pascal}, booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, pages = {679--687}, year = {2011}, editor = {Gordon, Geoffrey and Dunson, David and Dudík, Miroslav}, volume = {15}, series = {Proceedings of Machine Learning Research}, address = {Fort Lauderdale, FL, USA}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v15/sinn11a/sinn11a.pdf}, url = {https://proceedings.mlr.press/v15/sinn11a.html}, abstract = {In this theoretical paper we develop an asymptotic theory for Linear-Chain Conditional Random Fields (L-CRFs) and apply it to derive conditions under which the Maximum Likelihood Estimates (MLEs) of the model weights are strongly consistent. We first define L-CRFs for infinite sequences and analyze some of their basic properties. Then we establish conditions under which ergodicity of the observations implies ergodicity of the joint sequence of observations and labels. This result is the key ingredient to derive conditions for strong consistency of the MLEs. Interesting findings are that the consistency crucially depends on the limit behavior of the Hessian of the likelihood function and that, asymptotically, the state feature functions do not matter.} }
Endnote
%0 Conference Paper %T Asymptotic Theory for Linear-Chain Conditional Random Fields %A Mathieu Sinn %A Pascal Poupart %B Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2011 %E Geoffrey Gordon %E David Dunson %E Miroslav Dudík %F pmlr-v15-sinn11a %I PMLR %P 679--687 %U https://proceedings.mlr.press/v15/sinn11a.html %V 15 %X In this theoretical paper we develop an asymptotic theory for Linear-Chain Conditional Random Fields (L-CRFs) and apply it to derive conditions under which the Maximum Likelihood Estimates (MLEs) of the model weights are strongly consistent. We first define L-CRFs for infinite sequences and analyze some of their basic properties. Then we establish conditions under which ergodicity of the observations implies ergodicity of the joint sequence of observations and labels. This result is the key ingredient to derive conditions for strong consistency of the MLEs. Interesting findings are that the consistency crucially depends on the limit behavior of the Hessian of the likelihood function and that, asymptotically, the state feature functions do not matter.
RIS
TY - CPAPER TI - Asymptotic Theory for Linear-Chain Conditional Random Fields AU - Mathieu Sinn AU - Pascal Poupart BT - Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics DA - 2011/06/14 ED - Geoffrey Gordon ED - David Dunson ED - Miroslav Dudík ID - pmlr-v15-sinn11a PB - PMLR DP - Proceedings of Machine Learning Research VL - 15 SP - 679 EP - 687 L1 - http://proceedings.mlr.press/v15/sinn11a/sinn11a.pdf UR - https://proceedings.mlr.press/v15/sinn11a.html AB - In this theoretical paper we develop an asymptotic theory for Linear-Chain Conditional Random Fields (L-CRFs) and apply it to derive conditions under which the Maximum Likelihood Estimates (MLEs) of the model weights are strongly consistent. We first define L-CRFs for infinite sequences and analyze some of their basic properties. Then we establish conditions under which ergodicity of the observations implies ergodicity of the joint sequence of observations and labels. This result is the key ingredient to derive conditions for strong consistency of the MLEs. Interesting findings are that the consistency crucially depends on the limit behavior of the Hessian of the likelihood function and that, asymptotically, the state feature functions do not matter. ER -
APA
Sinn, M. & Poupart, P.. (2011). Asymptotic Theory for Linear-Chain Conditional Random Fields. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 15:679-687 Available from https://proceedings.mlr.press/v15/sinn11a.html.

Related Material