How often should I have to set oscilloscope probe's capacitance?
The probe capacitance adjustment is used to match the probe time constant, which consists of the probe capacitance and the probe resistance (usually 9 Megohms), to the time constant of the oscilloscope vertical amplifier input (which is usually 1 Megohm and 10 to 20 picofarads).  When the time constants match, the overall attenuation of the probe/oscilloscope will be 10 to 1 over a wide frequency range.  Normal practice in most labs is to adjust probes when they are first used both to ensure that they and the oscilloscope are operating OK and that nobody made any adjustments inbetween.  That adjustment should hold for a day or more unless the temperature changes drastically or the probe and/or oscilloscope are defective.  Defective could mean that the variable capacitor in the probe is not mechanically stable enough to hold its setting.