Best practices for masking image layers for compositing?
go to node editor and enable use nodes
add mask node and mix node
in uv editor change from paint to mask (if you want a mask image, upload that image to uv editor, the mask will be placed over your image)

LMB+Ctrl  to add point on the curve, close curve by enabling toggle cyclic
use this mask as input for mask node and plug mask node to factor of the mix node


for rotoscoping you could use this add on, see this for more info.