Set Matte not masking layers the same
Not sure about Set Matte, but you could try setting the transfer mode of the shape layer to Silhouette Alpha? That will cut out everything beneath that layer.

(You currently have the transfer modes hidden. Right click on the title bar above your layer names, click to Columns->Modes)