Title: Evaluation of Trace Alignment Quality and its Application in Medical Process Mining,
Abstract: Trace alignment algorithms have been used in process mining for discovering
the consensus treatment procedures and process deviations. Different alignment
algorithms, however, may produce very different results. No widely-adopted
method exists for evaluating the results of trace alignment. Existing
reference-free evaluation methods cannot adequately and comprehensively assess
the alignment quality. We analyzed and compared the existing evaluation
methods, identifying their limitations, and introduced improvements in two
reference-free evaluation methods. Our approach assesses the alignment result
globally instead of locally, and therefore helps the algorithm to optimize
overall alignment quality. We also introduced a novel metric to measure the
alignment complexity, which can be used as a constraint on alignment algorithm
optimization. We tested our evaluation methods on a trauma resuscitation
dataset and provided the medical explanation of the activities and patterns
identified as deviations using our proposed evaluation methods.