The direct tension pull-off test (ASTM D7522/D7522M-15, ASTM D4541-09e1, ASTM C1583/C1583M-13 and others) is the only practical, in situ test to assess bond strength of materials bonded to a concrete substrate relatively. Although the method is prescribed, there are a number of variations anecdotally seen in practice which may affect the uniform interpretation of the test method results. In particular, it was hypothesized that both shear and normal stresses introduced by the drilling process damage the sample affecting results disproportionately with the depth of cut. This study systematically investigated this hypothesis with the aim of providing improved guidance associated with depth, and indeed method, of core drilling. Preformed, hand-drilled, and core-drilled specimens were tested, and the results were compared. Additionally, a system for applying a controlled torsional moment to the specimens was used to quantitatively investigate the effect of such force being applied during the specimen preparation. This study also investigated the effect of using square cuts. Hand-drilled and core-drilled specimens were observed to have reduced pull-off capacity-exhibiting a reduction of 0.03 MPa/mm depth and 0.04 MPa/mm depth, respectively-with increased core depth, although this was not attributed to the torsion that may be present during the drilling process. Square specimens exhibited marginally lower capacity than comparable round specimens and a similar increased reduction in capacity of 0.05 MPa/mm depth of cut. Finally, a discussion of the implications and interpretation of pull-off test results is presented.