GPIO speed in LPC812 Cortex-M0 MCU from NXP
You have no way of KNOWING for a fact that your clock is what you think it is.  I recommend using a timer, with an interrupt triggered on reaching a count or overflow that simply toggles a bit to verify your system clock.

You should set the interrupt to be triggered at some reasonable rate -- maybe 1kHz.

Incorrect clock setup may or may not be your issue, but it is really a mandatory first step  in debugging such problems.  Many ARM processors have pretty complicated startups that involve changing clock modes when the clock is trying to do something but can't.