file_scan_directory() takes about 10 seconds to execute
It sounds like you're affected by a known issue in Drupal 7.


  Most probably, you are hitting Avoid re-scanning module directory when multiple modules are missing. It happens if you have some missing modules in your installation. Try to check your system table:

SELECT name, filename FROM system WHERE type = 'module' AND status = 1 ORDER BY filename

  
  And clean-up any modules that are still enabled but missing from the filesystem.
  
  Overall, Drupal 7 is way more resource friendly and scalable then Drupal 6, other than some unfortunate regressions like this one.


Looking at those functions, it does look like it's missing a module or maybe a single file of a module. Have a look at drupal_get_filename(), it does call the drupal_system_listing(), which calls this function if it can't find the requested file. Add a dpm(func_get_args()) right before it calls drupal_system_listing(), that should tell you which file it isn't finding.