WordPress: Creating new WP Users from Civi Contacts
You can use my CiviCRM WordPress Profile Sync plugin to do this, but I can't guarantee that it'll do them all in one go! You'd have to install the plugin, then set a PHP constant in the main plugin file (because it's a feature only really intended for developers at this stage) to enable this feature. Change line 22 to read:

define( 'CIVICRM_WP_PROFILE_SYNC_BULK', true );

EDIT: Better yet, add the above line to your wp-config.php file - then you won't have to re-edit the plugin after each update.

You will then be able to select contacts in CiviCRM and use the Bulk Operations menu to "Create WordPress Users from Contacts". I'd recommend limiting the number of contacts you migrate in each run. And that you backup before you try this.

Did I mention that you should backup before trying this? Oh, good!