Title: Test Case Prioritization Techniques for Model-Based Testing: A Replicated Study,
Abstract: Recently, several Test Case Prioritization (TCP) techniques have been
proposed to order test cases for achieving a goal during test execution,
particularly, revealing faults sooner. In the Model-Based Testing (MBT)
context, such techniques are usually based on heuristics related to structural
elements of the model and derived test cases. In this sense, techniques'
performance may vary due to a number of factors. While empirical studies
comparing the performance of TCP techniques have already been presented in
literature, there is still little knowledge, particularly in the MBT context,
about which factors may influence the outcomes suggested by a TCP technique. In
a previous family of empirical studies focusing on labeled transition systems,
we identified that the model layout, i.e. amount of branches, joins, and loops
in the model, alone may have little influence on the performance of TCP
techniques investigated, whereas characteristics of test cases that actually
fail definitely influences their performance. However, we considered only
synthetic artifacts in the study, which reduced the ability of representing
properly the reality. In this paper, we present a replication of one of these
studies, now with a larger and more representative selection of techniques and
considering test suites from industrial applications as experimental objects.
Our objective is to find out whether the results remain while increasing the
validity in comparison to the original study. Results reinforce that there is
no best performer among the investigated techniques and characteristics of test
cases that fail represent an important factor, although adaptive random based
techniques are less affected by it.