How does Blender know whether to use compositor output or 3D scene output
1. How does Blender know which to use? Are they separate outputs or are they one output?
They are separated thing. You can choose whether Blender to use Compositor and sequencer:

When you render, Blender save a bunch of information for camera (called render layers). Cameras are specified scene catchers in concept. Those information can be retrieved in Compositor interface for readable value, and for extra manipulate. Finally, compositor node can link to a final output (Composite), which is what you see in rendering window. (You can also choose what you see in view panel)

2. Is it possible to chroma key the scene?
Yes, totally possible to do that in Compositor. There is a node called keying, you can try it yourself.

(Notice that Blender knows about the original color of surface. Those hole in violin does not treat as black-like color in Keying, You can't do that in plain video)