How to delete a downloaded voice from Siri in iPhone5s
Go to Settings > General > Accessibility > VoiceOver > Speech and select the language. 

Then tap on Edit, swipe on the language and select Remove.