Remove disabled modules from Database
Yes, you can get errors when you delete the module files without disabling it first. It's best to completely uninstall the module, which allows the module to clean up after itself (eg. delete configuration from the database). After that, you can safely delete the files.

If you already deleted the files, the easiest way to get rid of bits and pieces left in the database, is to restore the files (download the module again and put the files back where they were), follow the uninstall process, and delete the files. Make sure you download the exact same version of the module you had installed earlier.

So how do you know which module version you had? Ideally you are already using a version control system (such as Git) to manage your code, so you can always look back in the history to find out which module version you were using. If you don't have version control, you can try to find the version number in the database as described in this answer. As a last resort, try to remember when you installed or last updated the module and look in the module's releases list to find a version close to that date.