Strange problem with Teensy 4.0 connected to external power only running code when serial connection is introduced
Do you have while(!Serial); in your code?

Also as mentioned - use capacitors on 7805, I'd suggest 1uF ceramics and maybe a bigger one if you have them too.