Broken buck converter investigation
The maximum input voltage and the minimum duty cycle for proper operation are determined in large part by the oscillator frequency.

The frequency you have chosen yields about 15.5% duty cycle vs. a minimum duty cycle (at 1MHz) of 13% (this is given by the minimum switch on time * the operating frequency - the higher the operating frequency, the higher the minimum duty cycle)

The maximum input is inversely proportional to the operating frequency, so a higher operating frequency yields a lower maximum Vin. The maximum input voltage at low loads for your circuit is about 24.4V (too close to the 24V you specify for comfort)

At input voltages above this, the output wanders:



Selecting a lower operating frequency will solve both issues.