How do I get gloss/matte control through an image texture?
I believe the issue is with your Image Texture node for your gloss map. Near the bottom of the node, change the Colour Space from sRGB to Non-color Data.

Edit

Your specular map is working properly. I checked by unplugged the normal map and looking at it. You seem to be having trouble getting your materials to look right, so I'm going to recommend that you watch the following two videos from BlenderGuru on how to create realistic materials. They're from pre 2.80, but it's still very helpful in understanding how all the settings affect your material.

How to Make Photorealistic PBR Materials - Part 1

How to Make Photorealistic PBR Materials - Part 2

BlenderGuru also linked this Google Doc in those videos, which essentially contains real-life values for various materials, it's worth bookmarking.

Physically Based Rendering Encyclopedia

PS: Part of getting reflective materials to look good is to have something to reflect...and not to have an overpowering Light.001 hidden from the viewport.