How to cancel email account setup without accepting server specified security policies?
Found it - you have to go through the phone's general settings, not the Email app.

Settings > Personal > Accounts > Exchange ActiveSync, then select your account, then there's a option "Remove" in the menu from the ... dots menu icon at the top right of the screen.