It should also be noted that signals can be recognized even when the model misclassifies relations, since ${\Delta }_s$ does not rely on correct classification: it merely quantifies the contribution of a word in context toward the correct label's score. If we examine the influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments based on what the system may tag as the second or third best class to choose. 
 Question: Where does proposed metric overlap with juman judgement?
influence of each word on the score of the correct relation, that impact should and does still correlate with human judgments