as he waited to take his leave of the castellan , he listened in on a conversation where the castellan and a sergeant-at-arms discussed the fact that leopold and hohenstaufen reached an agreement , and the prisoner was to be taken to speyer to meet with the emperor .