Parallel build for CI
Letting this sit in draft state for a while to see if issues pop up

Note that this doesn't affect Windows (not sure yet if it can). Also previous attempts had `-j4` but I believe we only get 2 virtual cores for the CI machines so that was probably overkill. 