What is just the plastic base of pin headers called?
I don't understand the stacking headers a bit too long comment. Also I was not able to find anything in https://www.digikey.ca/en/products/filter/rectangular-connectors-housings/319 or https://www.digikey.ca/en/products/filter/rectangular-connectors-accessories/340 matching this description, not that such a search would be on-topic - EESE is somewhat strict on the non-shopping policy.
Manufacturers of pin headers typically call this the insulator or housing, made out of nylon or glass-reinforced PBT thermoplastic.
In production I think it'd be rare to need this kind of thing - a spacer with no populated pins - since you'd want the pins prepopulated to reduce assembly labour.