Slow performance/Long response times
So something is definitely wrong; you should be seeing response times of around 300-500ms or so. What I'd suggest doing is profiling things on the Craft side as per the Profiling your Website with Craft CMS 3’s Debug Toolbar article.

You'll at least be able to see exactly what the bottlenecks are from a Craft perspective, and then see how much overhead is being added by the webserver.

If you find that it's slow on the Craft end of things, you may consider making sure that you are properly utilizing the cache tag