Tools for Empirically Analyzing AI Programs
Pre-proceedings of the Fifth International Workshop on Artificial Intelligence and Statistics, PMLR R0:35-41, 1995.
The paper describes two separate but synergistic tools for running experiments on large Lisp systems such as Artificial Intelligence planning systems, by which we mean systems that produce plans and execute them in some kind of simulator. The first tool, called CLIP (Common Lisp Instrumentation Package), allows the researcher to define and run experiments, including experimental conditions (parameter values of the planner or simulator) and data to be collected. The data are written out to data files that can be analyzed by statistics software. The second tool, called CLASP (Common Lisp Analytical Statistics Package), allows the researcher to analyze data from experiments by using graphics, statistical tests, and various kinds of data manipulation. CLASP has a graphical user interface (using CLIM, the Common Lisp Interface Manager, Version 2.0) and also allows data to be directly processed by Lisp functions. CLIP and CLASP form the foundation of a larger set of specialized tools we are building for the empirical analysis of AI programs.