I can only see vertex colors in Vertex Paint mode - otherwise gray
Figured out a workaround. These objects don't have any material in the glTF, just the vertex colors. So I created a material on one object, basic Principled BSDF, and as the color input I added Vertex Color node with the color set to COLOR_0. Then I linked that material to all the other objects, and it seems to work.

A bit painful, not sure why the importer can't just do that, but it does work for me.