Compositing Transparency / Layer Masking Problem
What you are experiencing is the result of antialiasing, more explicitly two overlapping masks with antialiasing, so the transparency of the upper mask is reveals some of the antialiasing of the underlaying mask.

You could get rid of anitaliasing it by switching off the AA samples using branched path tracing. That will give you perfect edges for the mask but it will affect the overall quality by having an aliased image that will look blocky...

 

A simpler and higher quality solution is not using a layer mask on the render layer for the grass.



Keep the layer mask on the layer for the bushes only.