Whilst the test accuracies have been measured for a baseline template-based CASP tool, templatefree models are also prone to misleading accuracy values. In both cases the task is to predict a series of viable outcomes, however the accuracy reflects the ability to predict the 'ground truth' from the underlying dataset, which inherently accounts for only one 'true' value, thus is partially known. In a similar work, Segler and Waller used the top 1, 10 and 50 accuracies to gauge the performance of their network, and showed that a model trained on 17,134 rules extracted from Reaxys, covering 52 % of the dataset, was able to predict the reaction center with accuracies of 50.1 %, 89.1 %, and 96 % respectively. In an extension of the work considering only single step reactions Baylon et. al. reported an accuracy of 81 % on 129 rules compared to 83 % on 137 rules by Segler and Waller. However, we have found that accuracy can be misleading when used for the assessment of overall model performance as shown in Figure and Figure , and specifically for the assessment of whether the network is able to correctly predict applicable reaction templates for single step reactions.