Is it possible to disable client side routing in Gatsby?

I'm using Gatsby to generate a static site which only has one page and will be served from AWS/S3. I'm running into an issue caused by Gatsby removing the object suffix from the URL (<https://s3.amazonaws.com/top-bucket/sub-bucket/index.html> becomes <https://s3.amazonaws.com/top-bucket/sub-bucket/>) after the page and the Gatsby runtime loads. This issue does not happen if I disable JavaScript, so I'm pretty certain it's caused by Gatsby's use of React/Reach Router. 

Is there any way to disable this behavior? I know I can probably setup a redirect on S3 to handle the request to the bucket, but I'd prefer to do this at the application level, if possible.