How to get video from webcam in a "Picture in Picture mode"
Have you tried starting a New Movie Recording first, selecting Float on Top from the view menu and then starting a New Screen Recording? Just tested it and it did a great job of a resizable PIP affect all in QT.