How to handle URLs like node/%.atom?
The code you wrote doesn't work because you are using a placeholder in the definition of the menu callback; such placeholders can be of two types:


anonymous placeholder. The placeholder is just %, which matches any string.
named placeholder. In this case, the placeholder is similar to %node, %user, used from Drupal core modules. If the placeholder is, for example, %node, Drupal will look for the node_load() function, and will pass it the string that in the path replaces the placeholder; when the function returns FALSE, then Drupal will return a 404 error page to the user. As the string following the % is used as name of the load function, to which the string "_load" is appended, characters that are not allowed in a function name are discarded (PHP cannot have a function with a name like .atom_load().


If your purpose is to define a menu callback for your module that returns an atom feed for the node, then I would define the callback as the following:

$items['node/%node/atom'] = array(
  'title' =&gt; 'atom node feed',
  'page callback' =&gt; 'atom_node_feed',
  'page arguments' =&gt; array(1),
  'access callback' =&gt; 'user_access',
  'access arguments' =&gt; array('generate atom feed'),
  'type' =&gt; MENU_CALLBACK,
);


The page callback will get the node object as first argument, not the node ID.

To create a path like http://drupalsite/node/20.atom, you need to create a path alias. 
You can assign path aliases using the administrative page at admin/config/search/path (Drupal 7) or admin/build/path (Drupal 6).
Using Drupal 7, a module can create a path alias calling path_save().
You can also use Pathauto to automatically set path aliases for nodes. The module uses tokens that allows to dynamically create path aliases that follows a defined schema. As far as I know, node path aliases are just set for paths like node/1; the path alias is not set for a path like node/1/edit, or node/1/atom.