When I bake a specular map out from Blender they always contain the glossy colors?
Bake the Specularity data, not the light interaction

According to the Wikipedia, specular maps contains the valuest that:


  represents the amount of reflectivity a surface has.


These maps are used in order to give to the rendering engine the information about how the light should interact when hitting that particular pixel of the surface.

What you should bake is the data about where a surface is capable of reflecting light and where is not, and not how the light interact with the surface (Glossy color, Glossy Direct, Glossy indirect, Combined....) because these are the result of the render engine.

Let's consider a simple material made of a Diffuse and Glossy shader mixed with a musgrave texture:



In this case it's easy to understand that the amount of Specularity is equal to the values that go into te mix shader factor. Where is black is Diffuse (no Specularity), where is white is Glossy (Specularity).

To bake that data just use an emission shader picking the color from the cable that is going into the mix shader's input socket



Here's the counter-prove. The baked map is now controlling the specularity of the surface as the musgrave was doing before.



For complex shaders things becomes harder, you'll have to build a custom nodetree capable of exctracting the Specularity data on each passage (e.g. if you are using multiple mix shaders).