# CLIP directional similarity: 0.0797976553440094 Like the CLIP Score, the higher the CLIP directional similarity, the better it is. It should be noted that the StableDiffusionInstructPix2PixPipeline exposes two arguments, namely, image_guidance_scale and guidance_scale that let you control the quality of the final edited image. We encourage you to experiment with these two arguments and see the impact of that on the directional similarity. We can extend the idea of this metric to measure how similar the original image and edited version are. To do that, we can just do F.cosine_similarity(img_feat_two, img_feat_one). For these kinds of edits, we would still want the primary semantics of the images to be preserved as much as possible, i.e., a high similarity score. We can use these metrics for similar pipelines such as the StableDiffusionPix2PixZeroPipeline. Both CLIP score and CLIP direction similarity rely on the CLIP model, which can make the evaluations biased. Extending metrics like IS, FID (discussed later), or KID can be difficult when the model under evaluation was pre-trained on a large image-captioning dataset (such as the LAION-5B dataset). This is because underlying these metrics is an InceptionNet (pre-trained on the ImageNet-1k dataset) used for extracting intermediate image features. The pre-training dataset of Stable Diffusion may have limited overlap with the pre-training dataset of InceptionNet, so it is not a good candidate here for feature extraction. Using the above metrics helps evaluate models that are class-conditioned. For example, DiT. It was pre-trained being conditioned on the ImageNet-1k classes.  Class-conditioned image generation Class-conditioned generative models are usually pre-trained on a class-labeled dataset such as ImageNet-1k. Popular metrics for evaluating these models include Fréchet Inception Distance (FID), Kernel Inception Distance (KID), and Inception Score (IS). In this document, we focus on FID (Heusel et al.). We show how to compute it with the DiTPipeline, which uses the DiT model under the hood. FID aims to measure how similar are two datasets of images. As per this resource: Fréchet Inception Distance is a measure of similarity between two datasets of images. It was shown to correlate well with the human judgment of visual quality and is most often used to evaluate the quality of samples of Generative Adversarial Networks. FID is calculated by computing the Fréchet distance between two Gaussians fitted to feature representations of the Inception network. These two datasets are essentially the dataset of real images and the dataset of fake images (generated images in our case). FID is usually calculated with two large datasets. However, for this document, we will work with two mini datasets. Let’s first download a few images from the ImageNet-1k training set:   Copied from zipfile import ZipFile