Use JSON instead HTML for access denied on some custom paths
Drupal returns the 404 in the format requested, so adding the query string ?_format=json to the request should return a JSON response under any circumstances. If Drupal doesn't detect the format it uses HTML by default. You could either help Drupal detecting the format in a request subscriber or implement an exception subscriber to return JSON exceptions similar to ExceptionJsonSubscriber::on4xx for your API paths.