A Comparison of Decision Analysis and Expert Rules for Sequential
  Diagnosis
  There has long been debate about the relative merits of decision theoretic
methods and heuristic rule-based approaches for reasoning under uncertainty. We
report an experimental comparison of the performance of the two approaches to
troubleshooting, specifically to test selection for fault diagnosis. We use as
experimental testbed the problem of diagnosing motorcycle engines. The first
approach employs heuristic test selection rules obtained from expert mechanics.
We compare it with the optimal decision analytic algorithm for test selection
which employs estimated component failure probabilities and test costs. The
decision analytic algorithm was found to reduce the expected cost (i.e. time)
to arrive at a diagnosis by an average of 14% relative to the expert rules.
Sensitivity analysis shows the results are quite robust to inaccuracy in the
probability and cost estimates. This difference suggests some interesting
implications for knowledge acquisition.
