Redirect from within controller always goes to home page
When you call a controller as error page via the basic site settings this is not the controller of the main request, this controller is executed in a subrequest to get a themed output of the otherwise simple text message you see for non 4xx exceptions like a 500 error. 

Better place the redirect in the main request, before the rendering of the exception is delegated to a subrequest by CustomPageExceptionHtmlSubscriber. You can use the default priority 0, because the core exception handler has -50.

@MarshallMorrise has posted the working code at https://drupal.stackexchange.com/a/290659/47547