Recipe to Make Pastel Ground/Primer with Marble Dust
I would look into a mixture of gum arabic dissolved in buttermilk, chalk whiting, a bit of almond oil and your marble dust, add your pigment and a drop of clove oil to keep the organic from going off.

500g gum arabic
1000ml buttermilk
250g - 500g chalk
250g - 500g marble dust
2-5g almond oil
Pigment
Clove oil


Bring buttermilk in a pot to about 80 degrees Celsius - don’t boil. Whisk in gum arabic until it is dissolved. Remove from heat. Add oils. Add chalk and marble dust until you get the thick paste you are comfortable with. Knead the paste like dough and until pigment is well mixed. Roll into sticks, let dry.

Don’t trust this recipe blindly, because chalk, pigment and gum arabic have different properties depending on purity and chemical composition. You’ll have to use your intuition and experience. You could use water instead of buttermilk. You could sump the chalk and marble in distilled water for 24 hours.