Example of hook_file_download
There is a couple of hooks in drupal 7 to restrict access to files.

One thing you have to realized is that these access hooks are only going to work if your download method is private, and the files are in your private path.

As you mention in your question there is a hook_file_download that should work in your case (assuming the file is private). This is not going to work if the file, someone is linking to, is somewhere public such as sites/default/files/abc123.txt.

If you happened to be uploading these private files in a field, then hook_file_download_access would seem more appropriate. This will give you the field that the file belongs to and also the related entity that this field is being used on so you should be able to do more complicated logic such as calling the the appropriate access function to check if the user has access to the field instead of having a hard-coded list in the code.