Is there a performance hit to using persistant variables?
In Drupal 6 and 7, variables created with variable_set() are stored in the {variable} database table. They are not automatically cleared when the cache is cleared. They are only removed when variable_del() is called.

Whenever a page is loaded, all variables in the {variable} table are loaded into the global $conf array. So yes - if a site is automatically creating new variables at a constant rate without cleaning them up, it could in theory cause performance degradation.

You can see in the variable_get() function that it is simply loading the global $conf array and looking for the requested variable within that.

It sounds like the custom code should have been using a dedicated database table for the data it was storing, because then it would only load that information when it needed to. Variables are loaded on every pageload.