How much the switching frequency change can impact the output voltage performance of a DC-DC converter?

Given the same output circuitry (switching MOSFETs, transformers, inductors etc.), the controller sees a small change by varying switching frequency, since the controller compensates for this change by adjusting the duty cycle. Thus, the output voltage will stay the same. Is my assumption correct?