Looking for the solution for very close board interconnect using pin headers
A thin spacer for this as a hack or refit and not a mass production method would be kapton or similar tape. Non conductive and high temperature. And a fixed height.
For mounting and conductivity  your are better off skipping the headers and use Castellations. Plated half holes soldered to matching pads on your board.