Why does my object lose color when the slot is changed?
It doesn't exactly "lose color". Paint slots are handled as layers, so if you add a new slot to your model it "spraypaints" the model with whatever colors you have in that slot. Since in your case there are no colors (or a texture, to be more precise) the model appears to have no color at all.