How can I individually scale the voronoi color chunks?
You are absolutely on the right track with the Voronoi textures - Just use 3 ones of different scale to get that random sized pebble effect. Clamp off some of the values with ColorRamps, so it's not too noisy, and when you multiply them together, use one as a mix factor (each time - hard to see on my second multiply node but its there) to reduce the influence of the overlaps.

The last step is to add a simple noise texture to give some color variance to the background: