• SFT minimises the difference between generated output z and target y w by optimising model π θ through negative log-likelihood (Eq. 2). • DPO A behaviour cloning (BC) regulariser is injected to reflect uniform output matching, with an additional SFT term in the final loss (Eq. 4). • KTO utilises nonpaired preference data D = {x (i) , y (i) , λ (i) } where λ denotes the desirability of y. The loss is computed from the generated output z in relation to a reference z ref and λ (Eq. 5).