How are user.tokens.inc and other similar files included? I notice that many core modules in Drupal 7 implement hook_token_info and hook_tokens in a hook.tokens.inc file, but I can't figure out how this file is included. Should I be doing the same with my own module? How can I be sure it will be loaded?
They do it by way of hook_hook_info():

Defines one or more hooks that are exposed by a module.
Normally hooks do not need to be explicitly defined. However, by declaring a hook explicitly, a module may define a &quot;group&quot; for it. Modules that implement a hook may then place their implementation in either $module.module or in $module.$group.inc. If the hook is located in $module.$group.inc, then that file will be automatically loaded when needed. In general, hooks that are rarely invoked and/or are very large should be placed in a separate include file, while hooks that are very short or very frequently called should be left in the main module file so that they are always available.

As luck would have it, the example from the docs shows exactly how the Token module does it:
function hook_hook_info() {
  $hooks['token_info'] = array(
    'group' =&gt; 'tokens',
  );
  $hooks['tokens'] = array(
    'group' =&gt; 'tokens',
  );
  return $hooks;
}


Should I be doing the same with my own module?

Totally up to you - it certainly can't hurt, other modules that need to implement your hook will just have an extra option for storing the hooks. But if you don't, they can still implement your hooks in the .module file anyway.