Using brush tool for alpha channels/rotoscoping in after-effects
The thing about masks is that you don't have to do them on every frame. To quickly do rotoscoping you let the computer do as much of the work as it can, by interpolating wherever possible.

To mask out the hands with the pen tool, draw the mask for the first frame and then move as far along the timeline as you can until the point where the motion dramatically changes, e.g. if the hand is moving up, continue until it stops or starts moving down. Adjust the mask points, trying to keep the points close to the part of the hand they were near when you first drew them. It's good to give yourself enough points to work with when you first draw the mask. 

Now go back between the two keyframes and find the point at which the maksk is the furthest from matching. Re-adjust, thus making a third keyframe. Now 
rinse, repeat: between each of your keyframes, pick the point where the mask is the worst and refine.

As you work you'll find that the mask needs less and less tweaking and eventually starts following the hand well. If your motion is really crazy this might not be until you've done every frame, but the adjustments you'll need to make will be smaller and smaller each time.

You'll find this will be tedious, but way less tedious than rendering as an image sequence and photoshoping each frame, or just opening the movie in photoshop (which is the answer to your question BTW). The difficulty in getting smooth motion that way would be immense. Think about the way hand drawn animation looks - the lines constantly 'boil' because each frame is painted from scratch. Painting the matte frame by frame will give you a result like that, which might be nice, but it doesn't seem like the look you're going for.

Put on a podcast or some music, make a coffee, get comfortable and get busy with the béziers.