How to figure which function(s) are triggered upon file download to check permissions
If the Drupal log shows you that the access has been denied, then you have a simple way to debug this: if you have a debugger, simply set a breakpoint in drupal_set_message. Skip calls to that function (there shouldn't be too many) until you get the one with the access denied message - and then look at the backtrace. This should show you which function denied the access.

If you don't have a debugger, you can simply (temporarily) add some extra code in that function to test for the relevant error message, and then output a backtrace using debug_backtrace. You can output the data on the screen, to a log file using drupal_debug (if you have the devel module enabled), add it to the generated error message, etc.