Why are vector quantizers not used in video codecs?
Theoretically, VQ is always better than scalar quantization (SQ). However, as you already mentioned, the complexity of VQ is higher than the complexity of SQ. It increases exponentially together with the dimensionality of vectors. Therefore, the question is how to design a practical coding system which can provide a best rate-distortion performance under a given complexity constraint.

A designing of a codebook is another very important issue in VQ. The main practical concerns about using a codebook in VQ are:


How to generate a universal codebook for a large number of images.
How to scale the codebook for various rate-distortion requirements.
How much storage space is reasonable for storing the codebook in memory.


All this practical constraints make it difficult for VQ to force through standardization process.