Do I have to unistall the disabled modules?
Disabled modules do not get executed so they should not affect performance but they may have variables and database tables that still exist. If you are not planning on using those modules again it is highly recommended to use the uninstall process at admin/build/modules/uninstall in D6 or admin/modules/uninstall in D7.