Voice memos not being deleted from iTunes
A workaround that I found, is to go to iTunes Music library, find the voice memo, and then manually delete it from there. After you sync, the Voice Memo playlist will be deleted from underneath your iPhone (in iTunes), so you won't see that file that had an exclamation mark anymore.