What is wrong with this simple node setup using textures from Poliigon?
You're using the normal map as if it was a displacement map. You have to put it through a normal map node first and then into the normal input of your diffuse shader. And inverting the gloss is actually right.

Refer to this: How to add a normal map texture and a diffuse texture on a model in cycles?.