Need neutral wire on "smart z-wave" wall switch - don't have one
My bet: you have two switch loops in the same box here, one old and one new
From what you've said so far, my bet is that you have not one, but two switch loops in this box.  Fortunately, while the switch loop for the old receptacle appears to be an &quot;old style&quot; loop with just a hot and a switched-hot, the new switch loop for the fan appears to be use the current Code configuration where neutral is brought down to the switch location alongside the hot and switched wires for precisely the reason you're asking this question.  So, I'd leave the black/white pair alone, and wire your switch as follows:

Hot/Line -- solo black wire
Switched/Load -- solo red wire
Neutral -- solo white wire
Ground -- bundle of bare wires in the back of the box

This should get the fan to work normally without disturbing the old receptacle circuit.