Is it possible to import "Assigned to" data for Activities?
I've used this CiviCRM Extension for just that purpose, however, it looks like it's only be released up to 4.4

Might be worth checking if there is a 4.6 version that's not been released

https://civicrm.org/extensions/api-csv-import-gui