Timeline isn't yellow above a section of a Multicam video
Looks like this one was a bug in Premiere. First, I noticed that the yellow bar was only missing for one of the clips that was part of the Multicam video. E.g. I noticed there were multiple "holes" in the timeline, but these only happened whenever the multicam sequence was trying to use "Video1". 

I went into Multicam edit more by right-clicking the Multicam video in the project editor, and choosing Open in Timeline.

Not sure what fixed it, but I toggled both the Toggle Track Output (eye icon) off then on again, and also Disabled and then Enabled the clip in question. This resolved the issue.