Why would taxonomy_term_save be slow when called from hook_install_tasks?
In this case, the problem resulted from having the Pathauto module enabled during installation.  When a new taxonomy term is saved, Drupal calls pathauto_taxonomy_term_insert(), which calls pathauto_taxonomy_term_update_alias(), which in turn calls pathauto_create_alias().  This last function calls an internal function, _pathauto_path_is_callback(), to determine whether an alias can be created for a given path, and this in turn calls the core function menu_get_item(), which contains the following lines:

if (variable_get('menu_rebuild_needed', FALSE) || !variable_get('menu_masks', array())) {
  menu_rebuild();
}


menu_rebuild contains the following lines:

if (defined('MAINTENANCE_MODE')) {
  variable_set('menu_rebuild_needed', TRUE);
}
else {
  variable_del('menu_rebuild_needed');
}


The cumulative effect seems to be that the menu_rebuild_needed variable is continually reset to TRUE, meaning that a complete menu rebuild is triggered each time a new taxonomy term is saved, slowing things down immensely (from ~100 terms saved per second to ~1 per second).

This seems like a bug, though it's not clear which component, if any, is at fault.  For now, I am working around the problem by disabling pathauto, creating the terms, and then re-enabling pathauto:

module_disable(array('pathauto'));
…
module_enable(array('pathauto'));