Is my understanding of how the convolution with stride 2 works in this example correct?
Yes, you're right, after the green one, it should also move two steps (because stride = 2) to the right once more. Note that in the $3 \times 3$ output volume picture, there's also still a white cell in the top right corner. That cell would get filled with whatever colour you choose to move to the right after the green one.
The blue one would then follow after what I described above, as the fourth square. I guess the author simply didn't feel like drawing a third square to the right of the green one, because the red + green squares already illustrate how the pattern works. The blue one was probably additionally drawn to illustrate that stride works the same way vertically as it does horizontally, e.g. blue also moves down two rows (because stride = 2).
See the bottom of the picture in the second answer here.