when upgrading should i delete all files AND all folders in templates_c directory?
Yes also the en_US subfolder. Actually especially that folder, since that's where the precompiled files get stored.

You can also use the menu option under Administer -> System Settings -> Cleanup Caches, which will do it for you.