How did Middle Chinese "hæwk" evolve into Mandarin "xué"?
I'm not an expert on historical Chinese phonology, but this doesn't seem like an uncharacteristically distant sound evolution compared to other words:
           Baxter       (Zhengzhang
           –Sagart       Shangfang)
Hanzi | Old Chinese | Middle Chinese | Mandarin
 學   |  *m-kˤruk   |      ɦˠʌk̚      |   xué
 穴   |  *[ɢ]ʷˤi[t] |      ɦwet̚      |   xué
 血   |  *m̥ˤik      |      hwet̚      |   xuè

Between Late Middle Chinese and Modern Mandarin, the final consonants /p/, /t/, /k/ were dropped. In Mandarin the only consonants a syllable can end on are n or ng.
/ɦˠ/ and /ɦw/ - back-of-the-mouth velarized and labialized fricatives, respectively, - likely shifted towards fricatives farther forward in the mouth.