Is probe compensation of a scope only for 10X setting?
No, you can compensate probes with any attenuation ratio.

The goal of probe compensation is to impedance match the probe to the scope. So, the waveform on screen is really the only indicator you have of a match.

Take a look at Table 2 in the Quick Start Guide for this switchable 10:1/1:1 probe:
https://literature.cdn.keysight.com/litweb/pdf/N2140-92002.pdf

There's a different input capacitance in 10:1 mode (100pf) and 1:1 mode (15pf). However, the compensation range is independent of the attenuation setting (15-30pf).

Probe compensation is something you can always consider. Very high end probes even ship with S-parameter data that gets auto-loaded into the scope for correction.

It's also worth noting that 1:1 probes give you lower bandwidth and higher loading.