How to texture paint only one part (normal map)?
At the bottom of the viewport next to the pivot options is a toggle for face select masking in Texture Paint mode. Turn that on, and either in Edit mode select the faces that are the offenders here, or in Texture Paint use the rmb to select the faces. You can sample the color from the normal map with the S key, and then paint it in masked by the face selection.