Unstable relay when using single power source
Try re-wiring like this and adding a 1000 uF capacitor: -

If this doesn't work then maybe your USB supply is dropping too low when the fan is activated. This could easily cause the circuit to reset.