Evaluation of selection in context-free grammar learning systems

Menno Zaanen, Nanne Noord
The 12th International Conference on Grammatical Inference, PMLR 34:193-206, 2014.

Abstract

Grammatical inference deals with learning of grammars describing languages. Formal grammatical inference aims at identifying families of languages that have a shared property, which can be used to prove efficient learnability of the families formally. In contrast, in empirical grammatical inference research, practical systems are developed that are applied to languages. The effectiveness of these systems is measured by comparing the learned grammar against a Gold standard which indicates the ground truth. From successful empirical learnability results, either shared properties may be identified, leading to further formal learnability results, or modifications to the systems may be made, improving practical results. Proper evaluation of empirical systems is, therefore, essential. Here, we evaluate and compare existing state-of-the-art context-free grammar learning systems (and novel systems based on combinations of existing phases) in a standardized evaluation environment (on a corpus of plain natural language sentences), illustrating future directions for empirical grammatical inference research.

Cite this Paper

BibTeX
@InProceedings{pmlr-v34-vanzaanen14a, title = {Evaluation of selection in context-free grammar learning systems}, author = {Zaanen, Menno and Noord, Nanne}, booktitle = {The 12th International Conference on Grammatical Inference}, pages = {193--206}, year = {2014}, editor = {Clark, Alexander and Kanazawa, Makoto and Yoshinaka, Ryo}, volume = {34}, series = {Proceedings of Machine Learning Research}, address = {Kyoto, Japan}, month = {17--19 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v34/vanzaanen14a.pdf}, url = {https://proceedings.mlr.press/v34/vanzaanen14a.html}, abstract = {Grammatical inference deals with learning of grammars describing languages. Formal grammatical inference aims at identifying families of languages that have a shared property, which can be used to prove efficient learnability of the families formally. In contrast, in empirical grammatical inference research, practical systems are developed that are applied to languages. The effectiveness of these systems is measured by comparing the learned grammar against a Gold standard which indicates the ground truth. From successful empirical learnability results, either shared properties may be identified, leading to further formal learnability results, or modifications to the systems may be made, improving practical results. Proper evaluation of empirical systems is, therefore, essential. Here, we evaluate and compare existing state-of-the-art context-free grammar learning systems (and novel systems based on combinations of existing phases) in a standardized evaluation environment (on a corpus of plain natural language sentences), illustrating future directions for empirical grammatical inference research.} }
Endnote
%0 Conference Paper %T Evaluation of selection in context-free grammar learning systems %A Menno Zaanen %A Nanne Noord %B The 12th International Conference on Grammatical Inference %C Proceedings of Machine Learning Research %D 2014 %E Alexander Clark %E Makoto Kanazawa %E Ryo Yoshinaka %F pmlr-v34-vanzaanen14a %I PMLR %P 193--206 %U https://proceedings.mlr.press/v34/vanzaanen14a.html %V 34 %X Grammatical inference deals with learning of grammars describing languages. Formal grammatical inference aims at identifying families of languages that have a shared property, which can be used to prove efficient learnability of the families formally. In contrast, in empirical grammatical inference research, practical systems are developed that are applied to languages. The effectiveness of these systems is measured by comparing the learned grammar against a Gold standard which indicates the ground truth. From successful empirical learnability results, either shared properties may be identified, leading to further formal learnability results, or modifications to the systems may be made, improving practical results. Proper evaluation of empirical systems is, therefore, essential. Here, we evaluate and compare existing state-of-the-art context-free grammar learning systems (and novel systems based on combinations of existing phases) in a standardized evaluation environment (on a corpus of plain natural language sentences), illustrating future directions for empirical grammatical inference research.
RIS
TY - CPAPER TI - Evaluation of selection in context-free grammar learning systems AU - Menno Zaanen AU - Nanne Noord BT - The 12th International Conference on Grammatical Inference DA - 2014/08/30 ED - Alexander Clark ED - Makoto Kanazawa ED - Ryo Yoshinaka ID - pmlr-v34-vanzaanen14a PB - PMLR DP - Proceedings of Machine Learning Research VL - 34 SP - 193 EP - 206 L1 - http://proceedings.mlr.press/v34/vanzaanen14a.pdf UR - https://proceedings.mlr.press/v34/vanzaanen14a.html AB - Grammatical inference deals with learning of grammars describing languages. Formal grammatical inference aims at identifying families of languages that have a shared property, which can be used to prove efficient learnability of the families formally. In contrast, in empirical grammatical inference research, practical systems are developed that are applied to languages. The effectiveness of these systems is measured by comparing the learned grammar against a Gold standard which indicates the ground truth. From successful empirical learnability results, either shared properties may be identified, leading to further formal learnability results, or modifications to the systems may be made, improving practical results. Proper evaluation of empirical systems is, therefore, essential. Here, we evaluate and compare existing state-of-the-art context-free grammar learning systems (and novel systems based on combinations of existing phases) in a standardized evaluation environment (on a corpus of plain natural language sentences), illustrating future directions for empirical grammatical inference research. ER -
APA
Zaanen, M. & Noord, N.. (2014). Evaluation of selection in context-free grammar learning systems. The 12th International Conference on Grammatical Inference, in Proceedings of Machine Learning Research 34:193-206 Available from https://proceedings.mlr.press/v34/vanzaanen14a.html.

Related Material