The study of artificial intelligence (AI) is indeed a strange pursuit. Unlike most other disciplines, few AI researchers even agree on a mutually acceptable definition of their chosen field of study. Some see AI as a sub field of computer science, others see AI as a computationally oriented branch of psychology or linguistics, while still others see it as a bag of tricks to be applied to an entire spectrum of diverse domains.
This lack of unified purpose among the AI community makes this a very exciting time for AI research: new and diverse projects are springing up literally every day. As one might imagine, however, this diversity also leads to genuine difficulties in assessing the significance and validity of AI research. These difficulties are an indication that AI has not yet matured as a science: it is still at the point where people are attempting to lay down (hopefully sound) foundations. Ritchie and Hanna [1] posit the following categorization as an aid in assessing the validity of an AI research endeavor:
(1) The project could introduce, in outline, a novel (or partly novel) idea or set of ideas.
(2) The project could elaborate the details of some approach. Starting with the kind of idea in (1), the research could criticize it or fill in further details
(3) The project could be an AI experiment, where a theory as in (1) and (2) is applied to some domain. Such experiments are usually computer programs that implement a particular theory.
As Ritchie and Hanna acknowledge, most AI work falls into categories (2) and (3). Such is the case here as well: credit for (1) should go to [2-6].
This book describes an AI experiment: a multiyear effort investigating the application of a novel machine-learning technique in a particular domain. Experience with the design and implementation of a computer program embodying these machine-learning ideas helps us in developing a more complete theory. What steps can be taken to insure that this work is a contribution to forward progress in the greater scheme of AI rather than a forgotten side trip? Ritchie and Hanna go on to note:
It is one of the peculiarities of AI that, although replication of practical results is a cornerstone of traditional science, it is rare to see published accounts of repetitions of AI work. It is not clear how to interpret this that those who do manage to repeat a published project do not regard this as publishable material. It may also be the case that an unsuccessful attempt at reimplementation would not be widely notified, since this might appear as an admission of incompetence. These circumstances impede the establishment of scientific standards within AI. Here then lies the key: one way to insure that this experiment was not in vain is to prepare this document with a view towards the rational reconstruction of the program.
To this end, we now make the following promises (largely inspired by [7] and [8]):
(1) To describe the theory behind the success of the system in a straightforward manner.
(2) To describe this system in as code-free a fashion as possible, preferring to revert to pseudo-code descriptions of important algorithms rather than reprinting the implementation!.
(3) To give annotated examples of the system in operation which give helpful insights into the operation of particular parts of the code.
(4) To avoid McDermott's third sin of AI research:
Only in a cautious footnote does he say, "the program was never actually finished," or, "only a preliminary version of the program was actually written."
All of the examples in this document are the product of a single version of the system, and are reproduced without embellishment. 2
Any judgement of the merits or success of this project must be based on the assumption that the technique here described (explanation-based learning) is a worthwhile addition to the machinelearning repertory. If one believes this to be the case, the validity of the work done and reported herein depends on meeting the following criteria:
(1) The description of the technique is thorough enough to give the reader a clear understanding of how it works.
(2) The experiment itself clearly establishes the relation between theory and practice (the implementation).
(3) The description of the program is thorough enough to permit the rational reconstruction and, therefore, the independent verification of this experiment.
If all of these criteria are met, then this research has accomplished its primary goal: to further elaborate and validate (by means of a prototype computer implementation in a nontrivial domain) a novel method of automatic knowledge acquisition.
This lack of unified purpose among the AI community makes this a very exciting time for AI research: new and diverse projects are springing up literally every day. As one might imagine, however, this diversity also leads to genuine difficulties in assessing the significance and validity of AI research. These difficulties are an indication that AI has not yet matured as a science: it is still at the point where people are attempting to lay down (hopefully sound) foundations. Ritchie and Hanna [1] posit the following categorization as an aid in assessing the validity of an AI research endeavor:
(1) The project could introduce, in outline, a novel (or partly novel) idea or set of ideas.
(2) The project could elaborate the details of some approach. Starting with the kind of idea in (1), the research could criticize it or fill in further details
(3) The project could be an AI experiment, where a theory as in (1) and (2) is applied to some domain. Such experiments are usually computer programs that implement a particular theory.
As Ritchie and Hanna acknowledge, most AI work falls into categories (2) and (3). Such is the case here as well: credit for (1) should go to [2-6].
This book describes an AI experiment: a multiyear effort investigating the application of a novel machine-learning technique in a particular domain. Experience with the design and implementation of a computer program embodying these machine-learning ideas helps us in developing a more complete theory. What steps can be taken to insure that this work is a contribution to forward progress in the greater scheme of AI rather than a forgotten side trip? Ritchie and Hanna go on to note:
It is one of the peculiarities of AI that, although replication of practical results is a cornerstone of traditional science, it is rare to see published accounts of repetitions of AI work. It is not clear how to interpret this that those who do manage to repeat a published project do not regard this as publishable material. It may also be the case that an unsuccessful attempt at reimplementation would not be widely notified, since this might appear as an admission of incompetence. These circumstances impede the establishment of scientific standards within AI. Here then lies the key: one way to insure that this experiment was not in vain is to prepare this document with a view towards the rational reconstruction of the program.
To this end, we now make the following promises (largely inspired by [7] and [8]):
(1) To describe the theory behind the success of the system in a straightforward manner.
(2) To describe this system in as code-free a fashion as possible, preferring to revert to pseudo-code descriptions of important algorithms rather than reprinting the implementation!.
(3) To give annotated examples of the system in operation which give helpful insights into the operation of particular parts of the code.
(4) To avoid McDermott's third sin of AI research:
Only in a cautious footnote does he say, "the program was never actually finished," or, "only a preliminary version of the program was actually written."
All of the examples in this document are the product of a single version of the system, and are reproduced without embellishment. 2
Any judgement of the merits or success of this project must be based on the assumption that the technique here described (explanation-based learning) is a worthwhile addition to the machinelearning repertory. If one believes this to be the case, the validity of the work done and reported herein depends on meeting the following criteria:
(1) The description of the technique is thorough enough to give the reader a clear understanding of how it works.
(2) The experiment itself clearly establishes the relation between theory and practice (the implementation).
(3) The description of the program is thorough enough to permit the rational reconstruction and, therefore, the independent verification of this experiment.
If all of these criteria are met, then this research has accomplished its primary goal: to further elaborate and validate (by means of a prototype computer implementation in a nontrivial domain) a novel method of automatic knowledge acquisition.
