is there a way i can control which parts of the render have the glow?
Use the threshold value.  

Pixels that are brighter than the threshold will get the glow.

Values lower than those set by the threshold will not be affected.

Read: Any way to control light halos in the compositor based on emission level?