DrawBench and PartiPrompts are prompt datasets used for qualitative benchmarking. DrawBench and PartiPrompts were introduced by Imagen and Parti respectively. From the official Parti website: PartiPrompts (P2) is a rich set of over 1600 prompts in English that we release as part of this work. P2 can be used to measure model capabilities across various categories and challenge aspects.  PartiPrompts has the following columns: Prompt Category of the prompt (such as “Abstract”, “World Knowledge”, etc.) Challenge reflecting the difficulty (such as “Basic”, “Complex”, “Writing & Symbols”, etc.) These benchmarks allow for side-by-side human evaluation of different image generation models. For this, the 🧨 Diffusers team has built Open Parti Prompts, which is a community-driven qualitative benchmark based on Parti Prompts to compare state-of-the-art open-source diffusion models: Open Parti Prompts Game: For 10 parti prompts, 4 generated images are shown and the user selects the image that suits the prompt best. Open Parti Prompts Leaderboard: The leaderboard comparing the currently best open-sourced diffusion models to each other. To manually compare images, let’s see how we can use diffusers on a couple of PartiPrompts. Below we show some prompts sampled across different challenges: Basic, Complex, Linguistic Structures, Imagination, and Writing & Symbols. Here we are using PartiPrompts as a dataset.   Copied from datasets import load_dataset