Message Length as an Effective Ockham’s Razor in Decision Tree Induction

Scott Needham, David L. Dowe
Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics, PMLR R3:216-223, 2001.

Abstract

The validity of the Ockham’s Razor principle is a topic of much debate. A series of empirical investigations have sought to discredit the principle by the application of decision trees to learning tasks using node cardinality as the objective function. As a response to these papers, we suggest that the message length of a hypothesis can be used as an effective interpretation of Ockham’s Razor, resulting in positive empirical support for the principle. The theoretical justification for this Bayesian interpretation is also investigated.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR3-needham01a, title = {Message Length as an Effective Ockham’s Razor in Decision Tree Induction}, author = {Needham, Scott and Dowe, David L.}, booktitle = {Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics}, pages = {216--223}, year = {2001}, editor = {Richardson, Thomas S. and Jaakkola, Tommi S.}, volume = {R3}, series = {Proceedings of Machine Learning Research}, month = {04--07 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r3/needham01a/needham01a.pdf}, url = {https://proceedings.mlr.press/r3/needham01a.html}, abstract = {The validity of the Ockham’s Razor principle is a topic of much debate. A series of empirical investigations have sought to discredit the principle by the application of decision trees to learning tasks using node cardinality as the objective function. As a response to these papers, we suggest that the message length of a hypothesis can be used as an effective interpretation of Ockham’s Razor, resulting in positive empirical support for the principle. The theoretical justification for this Bayesian interpretation is also investigated.}, note = {Reissued by PMLR on 31 March 2021.} }
Endnote
%0 Conference Paper %T Message Length as an Effective Ockham’s Razor in Decision Tree Induction %A Scott Needham %A David L. Dowe %B Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2001 %E Thomas S. Richardson %E Tommi S. Jaakkola %F pmlr-vR3-needham01a %I PMLR %P 216--223 %U https://proceedings.mlr.press/r3/needham01a.html %V R3 %X The validity of the Ockham’s Razor principle is a topic of much debate. A series of empirical investigations have sought to discredit the principle by the application of decision trees to learning tasks using node cardinality as the objective function. As a response to these papers, we suggest that the message length of a hypothesis can be used as an effective interpretation of Ockham’s Razor, resulting in positive empirical support for the principle. The theoretical justification for this Bayesian interpretation is also investigated. %Z Reissued by PMLR on 31 March 2021.
APA
Needham, S. & Dowe, D.L.. (2001). Message Length as an Effective Ockham’s Razor in Decision Tree Induction. Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R3:216-223 Available from https://proceedings.mlr.press/r3/needham01a.html. Reissued by PMLR on 31 March 2021.

Related Material