Load certain modules only if user is administrator
Modules are either enabled or disabled in Drupal. They then may use access control to determine whether the user has access to them.

If these are custom modules, you may wish to write them so that either the heavy processing is skipped for non-admins, or that the data is cached to avoid calculations on every page load.