, where x is the input, and y w and y l are the preferred and dispreferred outputs, respectively, with N being the total number of pairs in D. The goal is to learn an optimal function f : X ↔ Y via a model π θ parameterised by θ. We coordinate the molecule and caption generation tasks via instruction modelling .