What do colors mean in specular maps?
Light behaves differently on different materials. So to recreate these behaviours we use Maps and so we have different workflows. Mostly they are Metallic or Specular workflow. 

In specular workflow the "Specular" map defines the specular reflectivity of the material. And then the Materials are divided into 2 Categories Metals and Non Metals.

Specular Map in Metals are usually coloured which gives them the Reflective Color and there albedos are usually in greycscale.

Specular map in Non metals are usually in grey scale and the colour is defined by the albedo.   

Specular and metallic workflow guide 

Similar case study

The above Link will help you to clear your Concepts regarding all maps used in PBR texturing.