Blender only masks the outline of what I'm trying to mask
Ok, so it turns out that the direction of the mask was flipped. Selecting the mask and switching the direction fixed the problem.
For some reason I had this problem again and this didn't work. Maybe there are different ways to solve different problems with the same look or something, I don't know, its weird.