text: D Perplexity issues 
  
  In Section 5 we present perplexity as one of the metrics we use for evaluating the effect of our attack over the quality of 
  the generated response. However, perplexity is intended to measure the naturalness of text, and as such it is ill-suited for 
  comparing the quality of multiple natural texts. This results with the perplexity values of the responses of both the weak 
  and the strong model being close and withing the margin of error. Figure 7 shows the distribution of perplexity values of 
  the clean responses generated by both models, and the ROCAUC score computed on these two sets of values. As can be 
  seen, the perplexity values are quite similar between both models, with ROCAUC scores ranging between 0.38 to 0.47. 
  
  24 