Finishing sunken text in 3D prints
I think your instinct is good.  The trick is finding the right modeling compound, with the right curing properties, that will stick to your polymer dice, and will be available in colors you like.

Amazon has some epoxy-based clays that appear to be either terra-cotta or black.  If these colors work, this may be the right choice.  Most epoxies will cling well, and it seems to have minimal shrinkage when hardening.  You could probably add some pigment and alter the color, but I doubt you could get a white, yellow, or any light color.

There are some air-drying clays, but the reviews show that these have higher shrinkage.

There are some bake-to-harden clays that harden between 250 and 300 degrees Farenheit.  If you printed with ABS or a high temperature plastic, it might work.  You would have to try it to know.  300 F is only 149 C, so maybe.  PLA wouldn't stand a chance.  If you can print polycarbonate, it might work better.

This material looks interesting: https://www.amazon.com/Polymer-Modelling-Modeling-Tutorials-Accessories/dp/B0716D1VDM?ref_=fsclp_pl_dp_6  There are lots of colors, and it can be cured in a 275 Farenheit oven, with a hot-air gun, or by boiling in water.