Message Length as an Effective Ockham’s Razor in Decision Tree Induction
Proceedings of the Eighth International Workshop on Artificial Intelligence and Statistics, PMLR R3:216-223, 2001.
The validity of the Ockham’s Razor principle is a topic of much debate. A series of empirical investigations have sought to discredit the principle by the application of decision trees to learning tasks using node cardinality as the objective function. As a response to these papers, we suggest that the message length of a hypothesis can be used as an effective interpretation of Ockham’s Razor, resulting in positive empirical support for the principle. The theoretical justification for this Bayesian interpretation is also investigated.