Resize a live stream and overlay over another live stream for broadcasting
You can do this with OBS. I do it all the time.

The Mac, can with OBS take the feed(s) and arrange/resize them and then stream it to Wirecast(?, but not necessary) or directly to Youtube.

OBS handles both input from streams and from video cards, so you can have several inputs and combine them, very easily.