Basis of color index
What you may be missing is that no two telescope, filter and detector combinations are the same. A colour index defined by your method would be measured differently by different observers.

The solution is to define standards that  have defined magnitudes and colours that all observers can use to calibrate their observations.

In actual fact, the basic measurement is
$$ B-V = \alpha (b-v) + z + \beta X,$$
where $b$ and $v$ are native measured magnitudes (-2.5 log the detected adu with a correction for exposure time) and $X$ is the airmass. The constants $\alpha$, zeropoint $z$ and extinction coefficient $\beta$ are determined by observing a network of secondary standards with a range of colours at a range of airmasses.