Uninstalling modules doesn't remove them from modules list
The modules on the file system are not intended to be deleted if you uninstall them. Although I can see that the terminology may be confusing.

Uninstalling will delete any references to the module in your site (assuming the module is written to do so), but will not affect the file system in any way.

The only reference to the module will be on the modules page, it will not be used in any live pages on your site. So should be harmless, but if you don't want the module to appear on the modules page you will have to manually delete the files.