What happens if a user uploads an image but doesn't save the node? I would like to know what happens if a user uploads an image by image field but doesn't save that node?

It seems that image will be in the directory but no node refers to it. Is there anyway to remove such files?
The image files will be removed during cron runs.

This happens because the status column in file_managed for those files will still be zero, indicating that they're temporary, and can be safely removed after a period of time.

The responsible code is in system_cron(), in case you need to reproduce/analyse further.