Private filesystem does fire 403 on every role and file
If you need to debug access at the Drupal application level, then look at FileDownloadController::download; it's the controller method for the /system/files/{filepath} path; you can use your tool of choice to inspect the execution flow from request to the returned response object.
If the issue is outside the Drupal application level (e.g. web server, varnish) then it will require further stack troubleshooting.