What is the risk of leaving the MAN START PB 'ON' on the A320?
Well, documentation is a bit vague about this, and I suppose a technician would know more about the different things the ENG MAN START bp is doing, but from a quick glance at the FCOM there is one major difference that I can spot:


  [The FADEC durring a manual start]  (...) Closes the engine start valve, and cuts off the ignition when N2 reaches 43 %.


Bold emphasis is mine, no other conditions except N2>43%.

Leaving the ENG MAN START pb on means FADEC keeps assuming we are trying a manual start, and will cut off ignition anytime the N2 is above 43%. No prob as long as engines are running, right?

But that means (I assume) when you then enter turbulent/stormy weather in flight and turn on  IGN/START to prevent a flameout, nothing happens as FADEC is happy that N2 is above 43% and you manual start has been ‘succesfull’.

I don’t really know if it will even attempt an auto restart when you get a flameout, as the N2 will probably keep windmilling for some time depending on your airspeed, so you could end up with a big problem.