How do I stop getting reminders form a deleted Exchange account?
Try clearing your calendar data (Settings -> Applications -> Calendar -> Clear Data).

If that doesn't work then you might have to resort to a factory reset.

(If you have to go this route then there are lots of options for backing up your data before hand to minimize the pain)