had to “engineer” some prompts here.  Image-conditioned text-to-image generation In this case, we condition the generation pipeline with an input image as well as a text prompt. Let’s take the StableDiffusionInstructPix2PixPipeline, as an example. It takes an edit instruction as an input prompt and an input image to be edited. Here is one example:  One strategy to evaluate such a model is to measure the consistency of the change between the two images (in CLIP space) with the change between the two image captions (as shown in CLIP-Guided Domain Adaptation of Image Generators). This is referred to as the ”CLIP directional similarity“. Caption 1 corresponds to the input image (image 1) that is to be edited. Caption 2 corresponds to the edited image (image 2). It should reflect the edit instruction. Following is a pictorial overview:  We have prepared a mini dataset to implement this metric. Let’s first load the dataset.   Copied from datasets import load_dataset