ARM Cortex behavior on erased flash
The core will enter LOCKUP state. This failure is actually architecturally defined (v7M):


  Vector read error at reset, when reading initial PC or SP value
  Behavior Lockup at priority -1.Lockup address 0xFFFFFFFE


The core will then remain stuck in this state (since it is impossible to fetch from an XN region):


  In most cases, when the processor enters Lockup state, it remains in
  that state until a reset, such as from a watchdog.


This is in contrast to some of the other lockup states where the error can technically be cleared and the core will just keep trying forever.

I think that the SP being illegal is benign until the stack is used.