When removing a file it still is present on the machine
Permanent files have a status FILE_STATUS_PERMANENT.


  Indicates that the file is permanent and should not be deleted.
  
  Temporary files older than DRUPAL_MAXIMUM_TEMP_FILE_AGE will be
  removed during cron runs, but permanent files will not be removed
  during the file garbage collection process.


When you remove a file from a node, the file_managed record may simply be marked as status = 0 or anything other than FILE_PERMANENT, in which case, it's deleted during system_cron (if no entry in the file_usage table).

Basically, if you have cron running, it should be fine.