Deleting contacts on iCloud does not remove them from iPhone
It appears that the contacts that were not being deleted were linked to Google and Yahoo accounts. The data from them were imported to iCloud. I'm pretty sure iCloud was deleting the contacts from the iPhone that I deleted on the website/desktop. I went in to Settings, Accounts and disabled showing the contacts for Google and Yahoo and all the trashy contacts disappeared.