Is there a mistake in the Upgrading CivCRM for WordPress instructions?
The assumption that was made in the instructions is that you must extract the files into the /wp-content/plugins/ directory and then rename the resulting file to civicrm, just like the old directory was named. The previous step to this was to backup /wp-content/plugins/civicrm. 

E.g. Back up and delete /wp-content/plugins/civicrm then extract the new version and name it civicrm and put it in /wp-content/plugins/, thus replace the previous.