class_docstr: Pipeline for text-to-image generation using Stable Diffusion XL.
  
  This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the
  library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.)
  
  In addition the pipeline inherits the following loading methods:
      - *LoRA*: [`loaders.StableDiffusionXLLoraLoaderMixin.load_lora_weights`]
      - *Ckpt*: [`loaders.FromSingleFileMixin.from_single_file`]
  
  as well as the following saving methods:
      - *LoRA*: [`loaders.StableDiffusionXLLoraLoaderMixin.save_lora_weights`]
  
  Args:
      vae ([`AutoencoderKL`]):
          Variational Auto-Encoder (VAE) Model to encode and decode images to and from latent representations.
      text_encoder ([`CLIPTextModel`]):
          Frozen text-encoder. Stable Diffusion XL uses the text portion of
          [CLIP](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel), specifically
          the [clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14) variant.
      text_encoder_2 ([` CLIPTextModelWithProjection`]):
          Second frozen text-encoder. Stable Diffusion XL uses the text and pool portion of
          [CLIP](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModelWithProjection),
          specifically the
          [laion/CLIP-ViT-bigG-14-laion2B-39B-b160k](https://huggingface.co/laion/CLIP-ViT-bigG-14-laion2B-39B-b160k)
          variant.
      tokenizer (`CLIPTokenizer`):
          Tokenizer of class
          [CLIPTokenizer](https://huggingface.co/docs/transformers/v4.21.0/en/model_doc/clip#transformers.CLIPTokenizer).
      tokenizer_2 (`CLIPTokenizer`):
          Second Tokenizer of class
          [CLIPTokenizer](https://huggingface.co/docs/transformers/v4.21.0/en/model_doc/clip#transformers.CLIPTokenizer).
      unet ([`UNet2DConditionModel`]): Conditional U-Net architecture to denoise the encoded image latents.
      scheduler ([`SchedulerMixin`]):
          A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of
          [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`].
      requires_aesthetics_score (`bool`, *optional*, defaults to `"False"`):
          Whether the `unet` requires a aesthetic_score condition to be passed during inference. Also see the config
          of `stabilityai/stable-diffusion-xl-refiner-1-0`.
      force_zeros_for_empty_prompt (`bool`, *optional*, defaults to `"True"`):
          Whether the negative prompt embeddings shall be forced to always be set to 0. Also see the config of
          `stabilityai/stable-diffusion-xl-base-1-0`.
      add_watermarker (`bool`, *optional*):
          Whether to use the [invisible_watermark library](https://github.com/ShieldMnt/invisible-watermark/) to
          watermark output images. If not defined, it will default to True if the package is installed, otherwise no
          watermarker will be used.
docstr: Function invoked when calling the pipeline for generation.
  
  Args:
      prompt (`str` or `List[str]`, *optional*):
          The prompt or prompts to guide the image generation. If not defined, one has to pass `prompt_embeds`.
          instead.
      prompt_2 (`str` or `List[str]`, *optional*):
          The prompt or prompts to be sent to the `tokenizer_2` and `text_encoder_2`. If not defined, `prompt` is
          used in both text-encoders
      image (`PIL.Image.Image`):
          `Image`, or tensor representing an image batch which will be inpainted, *i.e.* parts of the image will
          be masked out with `mask_image` and repainted according to `prompt`.
      mask_image (`PIL.Image.Image`):
          `Image`, or tensor representing an image batch, to mask `image`. White pixels in the mask will be
          repainted, while black pixels will be preserved. If `mask_image` is a PIL image, it will be converted
          to a single channel (luminance) before use. If it's a tensor, it should contain one color channel (L)
          instead of 3, so the expected shape would be `(B, H, W, 1)`.
      height (`int`, *optional*, defaults to self.unet.config.sample_size * self.vae_scale_factor):
          The height in pixels of the generated image. This is set to 1024 by default for the best results.
          Anything below 512 pixels won't work well for
          [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
          and checkpoints that are not specifically fine-tuned on low resolutions.
      width (`int`, *optional*, defaults to self.unet.config.sample_size * self.vae_scale_factor):
          The width in pixels of the generated image. This is set to 1024 by default for the best results.
          Anything below 512 pixels won't work well for
          [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
          and checkpoints that are not specifically fine-tuned on low resolutions.
      strength (`float`, *optional*, defaults to 0.9999):
          Conceptually, indicates how much to transform the masked portion of the reference `image`. Must be
          between 0 and 1. `image` will be used as a starting point, adding more noise to it the larger the
          `strength`. The number of denoising steps depends on the amount of noise initially added. When
          `strength` is 1, added noise will be maximum and the denoising process will run for the full number of
          iterations specified in `num_inference_steps`. A value of 1, therefore, essentially ignores the masked
          portion of the reference `image`. Note that in the case of `denoising_start` being declared as an
          integer, the value of `strength` will be ignored.
      num_inference_steps (`int`, *optional*, defaults to 50):
          The number of denoising steps. More denoising steps usually lead to a higher quality image at the
          expense of slower inference.
      denoising_start (`float`, *optional*):
          When specified, indicates the fraction (between 0.0 and 1.0) of the total denoising process to be
          bypassed before it is initiated. Consequently, the initial part of the denoising process is skipped and
          it is assumed that the passed `image` is a partly denoised image. Note that when this is specified,
          strength will be ignored. The `denoising_start` parameter is particularly beneficial when this pipeline
          is integrated into a "Mixture of Denoisers" multi-pipeline setup, as detailed in [**Refining the Image
          Output**](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl#refining-the-image-output).
      denoising_end (`float`, *optional*):
          When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be
          completed before it is intentionally prematurely terminated. As a result, the returned sample will
          still retain a substantial amount of noise (ca. final 20% of timesteps still needed) and should be
          denoised by a successor pipeline that has `denoising_start` set to 0.8 so that it only denoises the
          final 20% of the scheduler. The denoising_end parameter should ideally be utilized when this pipeline
          forms a part of a "Mixture of Denoisers" multi-pipeline setup, as elaborated in [**Refining the Image
          Output**](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl#refining-the-image-output).
      guidance_scale (`float`, *optional*, defaults to 7.5):
          Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598).
          `guidance_scale` is defined as `w` of equation 2. of [Imagen
          Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale >
          1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`,
          usually at the expense of lower image quality.
      negative_prompt (`str` or `List[str]`, *optional*):
          The prompt or prompts not to guide the image generation. If not defined, one has to pass
          `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is
          less than `1`).
      negative_prompt_2 (`str` or `List[str]`, *optional*):
          The prompt or prompts not to guide the image generation to be sent to `tokenizer_2` and
          `text_encoder_2`. If not defined, `negative_prompt` is used in both text-encoders
      prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not
          provided, text embeddings will be generated from `prompt` input argument.
      negative_prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt
          weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input
          argument.
      pooled_prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting.
          If not provided, pooled text embeddings will be generated from `prompt` input argument.
      negative_pooled_prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt
          weighting. If not provided, pooled negative_prompt_embeds will be generated from `negative_prompt`
          input argument.
      num_images_per_prompt (`int`, *optional*, defaults to 1):
          The number of images to generate per prompt.
      eta (`float`, *optional*, defaults to 0.0):
          Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to
          [`schedulers.DDIMScheduler`], will be ignored for others.
      generator (`torch.Generator`, *optional*):
          One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html)
          to make generation deterministic.
      latents (`torch.FloatTensor`, *optional*):
          Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image
          generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
          tensor will ge generated by sampling using the supplied random `generator`.
      output_type (`str`, *optional*, defaults to `"pil"`):
          The output format of the generate image. Choose between
          [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`.
      return_dict (`bool`, *optional*, defaults to `True`):
          Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
          plain tuple.
      callback (`Callable`, *optional*):
          A function that will be called every `callback_steps` steps during inference. The function will be
          called with the following arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`.
      callback_steps (`int`, *optional*, defaults to 1):
          The frequency at which the `callback` function will be called. If not specified, the callback will be
          called at every step.
      cross_attention_kwargs (`dict`, *optional*):
          A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under
          `self.processor` in
          [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
      original_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
          If `original_size` is not the same as `target_size` the image will appear to be down- or upsampled.
          `original_size` defaults to `(height, width)` if not specified. Part of SDXL's micro-conditioning as
          explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
      crops_coords_top_left (`Tuple[int]`, *optional*, defaults to (0, 0)):
          `crops_coords_top_left` can be used to generate an image that appears to be "cropped" from the position
          `crops_coords_top_left` downwards. Favorable, well-centered images are usually achieved by setting
          `crops_coords_top_left` to (0, 0). Part of SDXL's micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
      target_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
          For most cases, `target_size` should be set to the desired height and width of the generated image. If
          not specified it will default to `(height, width)`. Part of SDXL's micro-conditioning as explained in
          section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
      negative_original_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
          To negatively condition the generation process based on a specific image resolution. Part of SDXL's
          micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
          information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
      negative_crops_coords_top_left (`Tuple[int]`, *optional*, defaults to (0, 0)):
          To negatively condition the generation process based on a specific crop coordinates. Part of SDXL's
          micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
          information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
      negative_target_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
          To negatively condition the generation process based on a target image resolution. It should be as same
          as the `target_size` for most cases. Part of SDXL's micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
          information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
      aesthetic_score (`float`, *optional*, defaults to 6.0):
          Used to simulate an aesthetic score of the generated image by influencing the positive text condition.
          Part of SDXL's micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
      negative_aesthetic_score (`float`, *optional*, defaults to 2.5):
          Part of SDXL's micro-conditioning as explained in section 2.2 of
          [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). Can be used to
          simulate an aesthetic score of the generated image by influencing the negative text condition.
      clip_skip (`int`, *optional*):
          Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that
          the output of the pre-final layer will be used for computing the prompt embeddings.
  
  Examples:
  
  Returns:
      [`~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput`] or `tuple`:
      [`~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput`] if `return_dict` is True, otherwise a
      `tuple. `tuple. When returning a tuple, the first element is a list with the generated images.
docstr_tok: ["Function","invoked","when","calling","the","pipeline","for","generation",".","Args",":","prompt","(","`","str","`","or","`","List","[","str","]","`",",","*","optional","*",")",":","The","prompt","or","prompts","to","guide","the","image","generation",".","If","not","defined",",","one","has","to","pass","`","prompt_embeds","`",".","instead",".","prompt_2","(","`","str","`","or","`","List","[","str","]","`",",","*","optional","*",")",":","The","prompt","or","prompts","to","be","sent","to","the","`","tokenizer_2","`","and","`","text_encoder_2","`",".","If","not","defined",",","`","prompt","`","is","used","in","both","text-encoders","image","(","`","PIL.Image.Image","`",")",":","`","Image","`",",","or","tensor","representing","an","image","batch","which","will","be","inpainted",",","*","i.e",".","*","parts","of","the","image","will","be","masked","out","with","`","mask_image","`","and","repainted","according","to","`","prompt","`",".","mask_image","(","`","PIL.Image.Image","`",")",":","`","Image","`",",","or","tensor","representing","an","image","batch",",","to","mask","`","image","`",".","White","pixels","in","the","mask","will","be","repainted",",","while","black","pixels","will","be","preserved",".","If","`","mask_image","`","is","a","PIL","image",",","it","will","be","converted","to","a","single","channel","(","luminance",")","before","use",".","If","it","'s","a","tensor",",","it","should","contain","one","color","channel","(","L",")","instead","of","3",",","so","the","expected","shape","would","be","`","(","B",",","H",",","W",",","1",")","`",".","height","(","`","int","`",",","*","optional","*",",","defaults","to","self.unet.config.sample_size","*","self.vae_scale_factor",")",":","The","height","in","pixels","of","the","generated","image",".","This","is","set","to","1024","by","default","for","the","best","results",".","Anything","below","512","pixels","wo","n't","work","well","for","[","stabilityai\/stable-diffusion-xl-base-1.0","]","(","https",":","\/\/huggingface.co\/stabilityai\/stable-diffusion-xl-base-1.0",")","and","checkpoints","that","are","not","specifically","fine-tuned","on","low","resolutions",".","width","(","`","int","`",",","*","optional","*",",","defaults","to","self.unet.config.sample_size","*","self.vae_scale_factor",")",":","The","width","in","pixels","of","the","generated","image",".","This","is","set","to","1024","by","default","for","the","best","results",".","Anything","below","512","pixels","wo","n't","work","well","for","[","stabilityai\/stable-diffusion-xl-base-1.0","]","(","https",":","\/\/huggingface.co\/stabilityai\/stable-diffusion-xl-base-1.0",")","and","checkpoints","that","are","not","specifically","fine-tuned","on","low","resolutions",".","strength","(","`","float","`",",","*","optional","*",",","defaults","to","0.9999",")",":","Conceptually",",","indicates","how","much","to","transform","the","masked","portion","of","the","reference","`","image","`",".","Must","be","between","0","and","1",".","`","image","`","will","be","used","as","a","starting","point",",","adding","more","noise","to","it","the","larger","the","`","strength","`",".","The","number","of","denoising","steps","depends","on","the","amount","of","noise","initially","added",".","When","`","strength","`","is","1",",","added","noise","will","be","maximum","and","the","denoising","process","will","run","for","the","full","number","of","iterations","specified","in","`","num_inference_steps","`",".","A","value","of","1",",","therefore",",","essentially","ignores","the","masked","portion","of","the","reference","`","image","`",".","Note","that","in","the","case","of","`","denoising_start","`","being","declared","as","an","integer",",","the","value","of","`","strength","`","will","be","ignored",".","num_inference_steps","(","`","int","`",",","*","optional","*",",","defaults","to","50",")",":","The","number","of","denoising","steps",".","More","denoising","steps","usually","lead","to","a","higher","quality","image","at","the","expense","of","slower","inference",".","denoising_start","(","`","float","`",",","*","optional","*",")",":","When","specified",",","indicates","the","fraction","(","between","0.0","and","1.0",")","of","the","total","denoising","process","to","be","bypassed","before","it","is","initiated",".","Consequently",",","the","initial","part","of","the","denoising","process","is","skipped","and","it","is","assumed","that","the","passed","`","image","`","is","a","partly","denoised","image",".","Note","that","when","this","is","specified",",","strength","will","be","ignored",".","The","`","denoising_start","`","parameter","is","particularly","beneficial","when","this","pipeline","is","integrated","into","a","``","Mixture","of","Denoisers","''","multi-pipeline","setup",",","as","detailed","in","[","*","*","Refining","the","Image","Output","*","*","]","(","https",":","\/\/huggingface.co\/docs\/diffusers\/api\/pipelines\/stable_diffusion\/stable_diffusion_xl","#","refining-the-image-output",")",".","denoising_end","(","`","float","`",",","*","optional","*",")",":","When","specified",",","determines","the","fraction","(","between","0.0","and","1.0",")","of","the","total","denoising","process","to","be","completed","before","it","is","intentionally","prematurely","terminated",".","As","a","result",",","the","returned","sample","will","still","retain","a","substantial","amount","of","noise","(","ca",".","final","20","%","of","timesteps","still","needed",")","and","should","be","denoised","by","a","successor","pipeline","that","has","`","denoising_start","`","set","to","0.8","so","that","it","only","denoises","the","final","20","%","of","the","scheduler",".","The","denoising_end","parameter","should","ideally","be","utilized","when","this","pipeline","forms","a","part","of","a","``","Mixture","of","Denoisers","''","multi-pipeline","setup",",","as","elaborated","in","[","*","*","Refining","the","Image","Output","*","*","]","(","https",":","\/\/huggingface.co\/docs\/diffusers\/api\/pipelines\/stable_diffusion\/stable_diffusion_xl","#","refining-the-image-output",")",".","guidance_scale","(","`","float","`",",","*","optional","*",",","defaults","to","7.5",")",":","Guidance","scale","as","defined","in","[","Classifier-Free","Diffusion","Guidance","]","(","https",":","\/\/arxiv.org\/abs\/2207.12598",")",".","`","guidance_scale","`","is","defined","as","`","w","`","of","equation","2.","of","[","Imagen","Paper","]","(","https",":","\/\/arxiv.org\/pdf\/2205.11487.pdf",")",".","Guidance","scale","is","enabled","by","setting","`","guidance_scale",">","1","`",".","Higher","guidance","scale","encourages","to","generate","images","that","are","closely","linked","to","the","text","`","prompt","`",",","usually","at","the","expense","of","lower","image","quality",".","negative_prompt","(","`","str","`","or","`","List","[","str","]","`",",","*","optional","*",")",":","The","prompt","or","prompts","not","to","guide","the","image","generation",".","If","not","defined",",","one","has","to","pass","`","negative_prompt_embeds","`","instead",".","Ignored","when","not","using","guidance","(","i.e.",",","ignored","if","`","guidance_scale","`","is","less","than","`","1","`",")",".","negative_prompt_2","(","`","str","`","or","`","List","[","str","]","`",",","*","optional","*",")",":","The","prompt","or","prompts","not","to","guide","the","image","generation","to","be","sent","to","`","tokenizer_2","`","and","`","text_encoder_2","`",".","If","not","defined",",","`","negative_prompt","`","is","used","in","both","text-encoders","prompt_embeds","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Pre-generated","text","embeddings",".","Can","be","used","to","easily","tweak","text","inputs",",","*","e.g",".","*","prompt","weighting",".","If","not","provided",",","text","embeddings","will","be","generated","from","`","prompt","`","input","argument",".","negative_prompt_embeds","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Pre-generated","negative","text","embeddings",".","Can","be","used","to","easily","tweak","text","inputs",",","*","e.g",".","*","prompt","weighting",".","If","not","provided",",","negative_prompt_embeds","will","be","generated","from","`","negative_prompt","`","input","argument",".","pooled_prompt_embeds","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Pre-generated","pooled","text","embeddings",".","Can","be","used","to","easily","tweak","text","inputs",",","*","e.g",".","*","prompt","weighting",".","If","not","provided",",","pooled","text","embeddings","will","be","generated","from","`","prompt","`","input","argument",".","negative_pooled_prompt_embeds","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Pre-generated","negative","pooled","text","embeddings",".","Can","be","used","to","easily","tweak","text","inputs",",","*","e.g",".","*","prompt","weighting",".","If","not","provided",",","pooled","negative_prompt_embeds","will","be","generated","from","`","negative_prompt","`","input","argument",".","num_images_per_prompt","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","number","of","images","to","generate","per","prompt",".","eta","(","`","float","`",",","*","optional","*",",","defaults","to","0.0",")",":","Corresponds","to","parameter","eta","(","\u03b7",")","in","the","DDIM","paper",":","https",":","\/\/arxiv.org\/abs\/2010.02502",".","Only","applies","to","[","`","schedulers.DDIMScheduler","`","]",",","will","be","ignored","for","others",".","generator","(","`","torch.Generator","`",",","*","optional","*",")",":","One","or","a","list","of","[","torch","generator","(","s",")","]","(","https",":","\/\/pytorch.org\/docs\/stable\/generated\/torch.Generator.html",")","to","make","generation","deterministic",".","latents","(","`","torch.FloatTensor","`",",","*","optional","*",")",":","Pre-generated","noisy","latents",",","sampled","from","a","Gaussian","distribution",",","to","be","used","as","inputs","for","image","generation",".","Can","be","used","to","tweak","the","same","generation","with","different","prompts",".","If","not","provided",",","a","latents","tensor","will","ge","generated","by","sampling","using","the","supplied","random","`","generator","`",".","output_type","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","pil","''","`",")",":","The","output","format","of","the","generate","image",".","Choose","between","[","PIL","]","(","https",":","\/\/pillow.readthedocs.io\/en\/stable\/",")",":","`","PIL.Image.Image","`","or","`","np.array","`",".","return_dict","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","or","not","to","return","a","[","`","~pipelines.stable_diffusion.StableDiffusionPipelineOutput","`","]","instead","of","a","plain","tuple",".","callback","(","`","Callable","`",",","*","optional","*",")",":","A","function","that","will","be","called","every","`","callback_steps","`","steps","during","inference",".","The","function","will","be","called","with","the","following","arguments",":","`","callback","(","step",":","int",",","timestep",":","int",",","latents",":","torch.FloatTensor",")","`",".","callback_steps","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","frequency","at","which","the","`","callback","`","function","will","be","called",".","If","not","specified",",","the","callback","will","be","called","at","every","step",".","cross_attention_kwargs","(","`","dict","`",",","*","optional","*",")",":","A","kwargs","dictionary","that","if","specified","is","passed","along","to","the","`","AttentionProcessor","`","as","defined","under","`","self.processor","`","in","[","diffusers.models.attention_processor","]","(","https",":","\/\/github.com\/huggingface\/diffusers\/blob\/main\/src\/diffusers\/models\/attention_processor.py",")",".","original_size","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","1024",",","1024",")",")",":","If","`","original_size","`","is","not","the","same","as","`","target_size","`","the","image","will","appear","to","be","down-","or","upsampled",".","`","original_size","`","defaults","to","`","(","height",",","width",")","`","if","not","specified",".","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","crops_coords_top_left","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","0",",","0",")",")",":","`","crops_coords_top_left","`","can","be","used","to","generate","an","image","that","appears","to","be","``","cropped","''","from","the","position","`","crops_coords_top_left","`","downwards",".","Favorable",",","well-centered","images","are","usually","achieved","by","setting","`","crops_coords_top_left","`","to","(","0",",","0",")",".","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","target_size","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","1024",",","1024",")",")",":","For","most","cases",",","`","target_size","`","should","be","set","to","the","desired","height","and","width","of","the","generated","image",".","If","not","specified","it","will","default","to","`","(","height",",","width",")","`",".","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","negative_original_size","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","1024",",","1024",")",")",":","To","negatively","condition","the","generation","process","based","on","a","specific","image","resolution",".","Part","of","SDXL's","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","For","more","information",",","refer","to","this","issue","thread",":","https",":","\/\/github.com\/huggingface\/diffusers\/issues\/4208",".","negative_crops_coords_top_left","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","0",",","0",")",")",":","To","negatively","condition","the","generation","process","based","on","a","specific","crop","coordinates",".","Part","of","SDXL's","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","For","more","information",",","refer","to","this","issue","thread",":","https",":","\/\/github.com\/huggingface\/diffusers\/issues\/4208",".","negative_target_size","(","`","Tuple","[","int","]","`",",","*","optional","*",",","defaults","to","(","1024",",","1024",")",")",":","To","negatively","condition","the","generation","process","based","on","a","target","image","resolution",".","It","should","be","as","same","as","the","`","target_size","`","for","most","cases",".","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","For","more","information",",","refer","to","this","issue","thread",":","https",":","\/\/github.com\/huggingface\/diffusers\/issues\/4208",".","aesthetic_score","(","`","float","`",",","*","optional","*",",","defaults","to","6.0",")",":","Used","to","simulate","an","aesthetic","score","of","the","generated","image","by","influencing","the","positive","text","condition",".","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","negative_aesthetic_score","(","`","float","`",",","*","optional","*",",","defaults","to","2.5",")",":","Part","of","SDXL","'s","micro-conditioning","as","explained","in","section","2.2","of","[","https",":","\/\/huggingface.co\/papers\/2307.01952","]","(","https",":","\/\/huggingface.co\/papers\/2307.01952",")",".","Can","be","used","to","simulate","an","aesthetic","score","of","the","generated","image","by","influencing","the","negative","text","condition",".","clip_skip","(","`","int","`",",","*","optional","*",")",":","Number","of","layers","to","be","skipped","from","CLIP","while","computing","the","prompt","embeddings",".","A","value","of","1","means","that","the","output","of","the","pre-final","layer","will","be","used","for","computing","the","prompt","embeddings",".","Examples",":","Returns",":","[","`","~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput","`","]","or","`","tuple","`",":","[","`","~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput","`","]","if","`","return_dict","`","is","True",",","otherwise","a","`","tuple",".","`","tuple",".","When","returning","a","tuple",",","the","first","element","is","a","list","with","the","generated","images","."]
code: def __call__(
      self,
      prompt: Union[str, List[str]] = None,
      prompt_2: Optional[Union[str, List[str]]] = None,
      image: PipelineImageInput = None,
      mask_image: PipelineImageInput = None,
      masked_image_latents: torch.FloatTensor = None,
      height: Optional[int] = None,
      width: Optional[int] = None,
      strength: float = 0.9999,
      num_inference_steps: int = 50,
      denoising_start: Optional[float] = None,
      denoising_end: Optional[float] = None,
      guidance_scale: float = 7.5,
      negative_prompt: Optional[Union[str, List[str]]] = None,
      negative_prompt_2: Optional[Union[str, List[str]]] = None,
      num_images_per_prompt: Optional[int] = 1,
      eta: float = 0.0,
      generator: Optional[
          Union[torch.Generator, List[torch.Generator]]
      ] = None,
      latents: Optional[torch.FloatTensor] = None,
      prompt_embeds: Optional[torch.FloatTensor] = None,
      negative_prompt_embeds: Optional[torch.FloatTensor] = None,
      pooled_prompt_embeds: Optional[torch.FloatTensor] = None,
      negative_pooled_prompt_embeds: Optional[torch.FloatTensor] = None,
      output_type: Optional[str] = "pil",
      return_dict: bool = True,
      callback: Optional[
          Callable[[int, int, torch.FloatTensor], None]
      ] = None,
      callback_steps: int = 1,
      cross_attention_kwargs: Optional[Dict[str, Any]] = None,
      guidance_rescale: float = 0.0,
      original_size: Tuple[int, int] = None,
      crops_coords_top_left: Tuple[int, int] = (0, 0),
      target_size: Tuple[int, int] = None,
      negative_original_size: Optional[Tuple[int, int]] = None,
      negative_crops_coords_top_left: Tuple[int, int] = (0, 0),
      negative_target_size: Optional[Tuple[int, int]] = None,
      aesthetic_score: float = 6.0,
      negative_aesthetic_score: float = 2.5,
      clip_skip: Optional[int] = None,
  ):
      r"""
      Function invoked when calling the pipeline for generation.
  
      Args:
          prompt (`str` or `List[str]`, *optional*):
              The prompt or prompts to guide the image generation. If not defined, one has to pass `prompt_embeds`.
              instead.
          prompt_2 (`str` or `List[str]`, *optional*):
              The prompt or prompts to be sent to the `tokenizer_2` and `text_encoder_2`. If not defined, `prompt` is
              used in both text-encoders
          image (`PIL.Image.Image`):
              `Image`, or tensor representing an image batch which will be inpainted, *i.e.* parts of the image will
              be masked out with `mask_image` and repainted according to `prompt`.
          mask_image (`PIL.Image.Image`):
              `Image`, or tensor representing an image batch, to mask `image`. White pixels in the mask will be
              repainted, while black pixels will be preserved. If `mask_image` is a PIL image, it will be converted
              to a single channel (luminance) before use. If it's a tensor, it should contain one color channel (L)
              instead of 3, so the expected shape would be `(B, H, W, 1)`.
          height (`int`, *optional*, defaults to self.unet.config.sample_size * self.vae_scale_factor):
              The height in pixels of the generated image. This is set to 1024 by default for the best results.
              Anything below 512 pixels won't work well for
              [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
              and checkpoints that are not specifically fine-tuned on low resolutions.
          width (`int`, *optional*, defaults to self.unet.config.sample_size * self.vae_scale_factor):
              The width in pixels of the generated image. This is set to 1024 by default for the best results.
              Anything below 512 pixels won't work well for
              [stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
              and checkpoints that are not specifically fine-tuned on low resolutions.
          strength (`float`, *optional*, defaults to 0.9999):
              Conceptually, indicates how much to transform the masked portion of the reference `image`. Must be
              between 0 and 1. `image` will be used as a starting point, adding more noise to it the larger the
              `strength`. The number of denoising steps depends on the amount of noise initially added. When
              `strength` is 1, added noise will be maximum and the denoising process will run for the full number of
              iterations specified in `num_inference_steps`. A value of 1, therefore, essentially ignores the masked
              portion of the reference `image`. Note that in the case of `denoising_start` being declared as an
              integer, the value of `strength` will be ignored.
          num_inference_steps (`int`, *optional*, defaults to 50):
              The number of denoising steps. More denoising steps usually lead to a higher quality image at the
              expense of slower inference.
          denoising_start (`float`, *optional*):
              When specified, indicates the fraction (between 0.0 and 1.0) of the total denoising process to be
              bypassed before it is initiated. Consequently, the initial part of the denoising process is skipped and
              it is assumed that the passed `image` is a partly denoised image. Note that when this is specified,
              strength will be ignored. The `denoising_start` parameter is particularly beneficial when this pipeline
              is integrated into a "Mixture of Denoisers" multi-pipeline setup, as detailed in [**Refining the Image
              Output**](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl#refining-the-image-output).
          denoising_end (`float`, *optional*):
              When specified, determines the fraction (between 0.0 and 1.0) of the total denoising process to be
              completed before it is intentionally prematurely terminated. As a result, the returned sample will
              still retain a substantial amount of noise (ca. final 20% of timesteps still needed) and should be
              denoised by a successor pipeline that has `denoising_start` set to 0.8 so that it only denoises the
              final 20% of the scheduler. The denoising_end parameter should ideally be utilized when this pipeline
              forms a part of a "Mixture of Denoisers" multi-pipeline setup, as elaborated in [**Refining the Image
              Output**](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl#refining-the-image-output).
          guidance_scale (`float`, *optional*, defaults to 7.5):
              Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598).
              `guidance_scale` is defined as `w` of equation 2. of [Imagen
              Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale >
              1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`,
              usually at the expense of lower image quality.
          negative_prompt (`str` or `List[str]`, *optional*):
              The prompt or prompts not to guide the image generation. If not defined, one has to pass
              `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is
              less than `1`).
          negative_prompt_2 (`str` or `List[str]`, *optional*):
              The prompt or prompts not to guide the image generation to be sent to `tokenizer_2` and
              `text_encoder_2`. If not defined, `negative_prompt` is used in both text-encoders
          prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not
              provided, text embeddings will be generated from `prompt` input argument.
          negative_prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt
              weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input
              argument.
          pooled_prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting.
              If not provided, pooled text embeddings will be generated from `prompt` input argument.
          negative_pooled_prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt
              weighting. If not provided, pooled negative_prompt_embeds will be generated from `negative_prompt`
              input argument.
          num_images_per_prompt (`int`, *optional*, defaults to 1):
              The number of images to generate per prompt.
          eta (`float`, *optional*, defaults to 0.0):
              Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to
              [`schedulers.DDIMScheduler`], will be ignored for others.
          generator (`torch.Generator`, *optional*):
              One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html)
              to make generation deterministic.
          latents (`torch.FloatTensor`, *optional*):
              Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image
              generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
              tensor will ge generated by sampling using the supplied random `generator`.
          output_type (`str`, *optional*, defaults to `"pil"`):
              The output format of the generate image. Choose between
              [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`.
          return_dict (`bool`, *optional*, defaults to `True`):
              Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
              plain tuple.
          callback (`Callable`, *optional*):
              A function that will be called every `callback_steps` steps during inference. The function will be
              called with the following arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`.
          callback_steps (`int`, *optional*, defaults to 1):
              The frequency at which the `callback` function will be called. If not specified, the callback will be
              called at every step.
          cross_attention_kwargs (`dict`, *optional*):
              A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under
              `self.processor` in
              [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
          original_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
              If `original_size` is not the same as `target_size` the image will appear to be down- or upsampled.
              `original_size` defaults to `(height, width)` if not specified. Part of SDXL's micro-conditioning as
              explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
          crops_coords_top_left (`Tuple[int]`, *optional*, defaults to (0, 0)):
              `crops_coords_top_left` can be used to generate an image that appears to be "cropped" from the position
              `crops_coords_top_left` downwards. Favorable, well-centered images are usually achieved by setting
              `crops_coords_top_left` to (0, 0). Part of SDXL's micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
          target_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
              For most cases, `target_size` should be set to the desired height and width of the generated image. If
              not specified it will default to `(height, width)`. Part of SDXL's micro-conditioning as explained in
              section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
          negative_original_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
              To negatively condition the generation process based on a specific image resolution. Part of SDXL's
              micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
              information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
          negative_crops_coords_top_left (`Tuple[int]`, *optional*, defaults to (0, 0)):
              To negatively condition the generation process based on a specific crop coordinates. Part of SDXL's
              micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
              information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
          negative_target_size (`Tuple[int]`, *optional*, defaults to (1024, 1024)):
              To negatively condition the generation process based on a target image resolution. It should be as same
              as the `target_size` for most cases. Part of SDXL's micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). For more
              information, refer to this issue thread: https://github.com/huggingface/diffusers/issues/4208.
          aesthetic_score (`float`, *optional*, defaults to 6.0):
              Used to simulate an aesthetic score of the generated image by influencing the positive text condition.
              Part of SDXL's micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952).
          negative_aesthetic_score (`float`, *optional*, defaults to 2.5):
              Part of SDXL's micro-conditioning as explained in section 2.2 of
              [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). Can be used to
              simulate an aesthetic score of the generated image by influencing the negative text condition.
          clip_skip (`int`, *optional*):
              Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that
              the output of the pre-final layer will be used for computing the prompt embeddings.
  
      Examples:
  
      Returns:
          [`~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput`] or `tuple`:
          [`~pipelines.stable_diffusion.StableDiffusionXLPipelineOutput`] if `return_dict` is True, otherwise a
          `tuple. `tuple. When returning a tuple, the first element is a list with the generated images.
      """
      # 0. Default height and width to unet
      height = (
          height or self.unet.config.sample_size * self.vae_scale_factor
      )
      width = (
          width or self.unet.config.sample_size * self.vae_scale_factor
      )
  
      # 1. Check inputs
      self.check_inputs(
          prompt,
          prompt_2,
          height,
          width,
          strength,
          callback_steps,
          negative_prompt,
          negative_prompt_2,
          prompt_embeds,
          negative_prompt_embeds,
      )
  
      # 2. Define call parameters
      if prompt is not None and isinstance(prompt, str):
          batch_size = 1
      elif prompt is not None and isinstance(prompt, list):
          batch_size = len(prompt)
      else:
          batch_size = prompt_embeds.shape[0]
  
      device = self._execution_device
      # here `guidance_scale` is defined analog to the guidance weight `w` of equation (2)
      # of the Imagen paper: https://arxiv.org/pdf/2205.11487.pdf . `guidance_scale = 1`
      # corresponds to doing no classifier free guidance.
      do_classifier_free_guidance = guidance_scale > 1.0
  
      # 3. Encode input prompt
      text_encoder_lora_scale = (
          cross_attention_kwargs.get("scale", None)
          if cross_attention_kwargs is not None
          else None
      )
  
      (
          prompt_embeds,
          negative_prompt_embeds,
          pooled_prompt_embeds,
          negative_pooled_prompt_embeds,
      ) = self.encode_prompt(
          prompt=prompt,
          prompt_2=prompt_2,
          device=device,
          num_images_per_prompt=num_images_per_prompt,
          do_classifier_free_guidance=do_classifier_free_guidance,
          negative_prompt=negative_prompt,
          negative_prompt_2=negative_prompt_2,
          prompt_embeds=prompt_embeds,
          negative_prompt_embeds=negative_prompt_embeds,
          pooled_prompt_embeds=pooled_prompt_embeds,
          negative_pooled_prompt_embeds=negative_pooled_prompt_embeds,
          lora_scale=text_encoder_lora_scale,
          clip_skip=clip_skip,
      )
  
      # 4. set timesteps
      def denoising_value_valid(dnv):
          return isinstance(denoising_end, float) and 0 < dnv < 1
  
      self.scheduler.set_timesteps(num_inference_steps, device=device)
      timesteps, num_inference_steps = self.get_timesteps(
          num_inference_steps,
          strength,
          device,
          denoising_start=denoising_start
          if denoising_value_valid
          else None,
      )
      # check that number of inference steps is not < 1 - as this doesn't make sense
      if num_inference_steps < 1:
          raise ValueError(
              f"After adjusting the num_inference_steps by strength parameter: {strength}, the number of pipeline"
              f"steps is {num_inference_steps} which is < 1 and not appropriate for this pipeline."
          )
      # at which timestep to set the initial noise (n.b. 50% if strength is 0.5)
      latent_timestep = timesteps[:1].repeat(
          batch_size * num_images_per_prompt
      )
      # create a boolean to check if the strength is set to 1. if so then initialise the latents with pure noise
      is_strength_max = strength == 1.0
  
      # 5. Preprocess mask and image
      init_image = self.image_processor.preprocess(
          image, height=height, width=width
      )
      init_image = init_image.to(dtype=torch.float32)
  
      mask = self.mask_processor.preprocess(
          mask_image, height=height, width=width
      )
  
      if masked_image_latents is not None:
          masked_image = masked_image_latents
      elif init_image.shape[1] == 4:
          # if images are in latent space, we can't mask it
          masked_image = None
      else:
          masked_image = init_image * (mask < 0.5)
  
      # 6. Prepare latent variables
      num_channels_latents = self.vae.config.latent_channels
      num_channels_unet = self.unet.config.in_channels
      return_image_latents = num_channels_unet == 4
  
      add_noise = True if denoising_start is None else False
      latents_outputs = self.prepare_latents(
          batch_size * num_images_per_prompt,
          num_channels_latents,
          height,
          width,
          prompt_embeds.dtype,
          device,
          generator,
          latents,
          image=init_image,
          timestep=latent_timestep,
          is_strength_max=is_strength_max,
          add_noise=add_noise,
          return_noise=True,
          return_image_latents=return_image_latents,
      )
  
      if return_image_latents:
          latents, noise, image_latents = latents_outputs
      else:
          latents, noise = latents_outputs
  
      # 7. Prepare mask latent variables
      mask, masked_image_latents = self.prepare_mask_latents(
          mask,
          masked_image,
          batch_size * num_images_per_prompt,
          height,
          width,
          prompt_embeds.dtype,
          device,
          generator,
          do_classifier_free_guidance,
      )
  
      # 8. Check that sizes of mask, masked image and latents match
      if num_channels_unet == 9:
          # default case for runwayml/stable-diffusion-inpainting
          num_channels_mask = mask.shape[1]
          num_channels_masked_image = masked_image_latents.shape[1]
          if (
              num_channels_latents
              + num_channels_mask
              + num_channels_masked_image
              != self.unet.config.in_channels
          ):
              raise ValueError(
                  f"Incorrect configuration settings! The config of `pipeline.unet`: {self.unet.config} expects"
                  f" {self.unet.config.in_channels} but received `num_channels_latents`: {num_channels_latents} +"
                  f" `num_channels_mask`: {num_channels_mask} + `num_channels_masked_image`: {num_channels_masked_image}"
                  f" = {num_channels_latents+num_channels_masked_image+num_channels_mask}. Please verify the config of"
                  " `pipeline.unet` or your `mask_image` or `image` input."
              )
      elif num_channels_unet != 4:
          raise ValueError(
              f"The unet {self.unet.__class__} should have either 4 or 9 input channels, not {self.unet.config.in_channels}."
          )
      # 8.1 Prepare extra step kwargs.
      extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
  
      # 9. Prepare extra step kwargs. TODO: Logic should ideally just be moved out of the pipeline
      height, width = latents.shape[-2:]
      height = height * self.vae_scale_factor
      width = width * self.vae_scale_factor
  
      original_size = original_size or (height, width)
      target_size = target_size or (height, width)
  
      # 10. Prepare added time ids & embeddings
      if negative_original_size is None:
          negative_original_size = original_size
      if negative_target_size is None:
          negative_target_size = target_size
  
      add_text_embeds = pooled_prompt_embeds
      add_time_ids, add_neg_time_ids = self._get_add_time_ids(
          original_size,
          crops_coords_top_left,
          target_size,
          aesthetic_score,
          negative_aesthetic_score,
          negative_original_size,
          negative_crops_coords_top_left,
          negative_target_size,
          dtype=prompt_embeds.dtype,
      )
      add_time_ids = add_time_ids.repeat(
          batch_size * num_images_per_prompt, 1
      )
  
      if do_classifier_free_guidance:
          prompt_embeds = torch.cat(
              [negative_prompt_embeds, prompt_embeds], dim=0
          )
          add_text_embeds = torch.cat(
              [negative_pooled_prompt_embeds, add_text_embeds], dim=0
          )
          add_neg_time_ids = add_neg_time_ids.repeat(
              batch_size * num_images_per_prompt, 1
          )
          add_time_ids = torch.cat(
              [add_neg_time_ids, add_time_ids], dim=0
          )
  
      prompt_embeds = prompt_embeds.to(device)
      add_text_embeds = add_text_embeds.to(device)
      add_time_ids = add_time_ids.to(device)
  
      # 11. Denoising loop
      num_warmup_steps = max(
          len(timesteps) - num_inference_steps * self.scheduler.order, 0
      )
  
      if (
          denoising_end is not None
          and denoising_start is not None
          and denoising_value_valid(denoising_end)
          and denoising_value_valid(denoising_start)
          and denoising_start >= denoising_end
      ):
          raise ValueError(
              f"`denoising_start`: {denoising_start} cannot be larger than or equal to `denoising_end`: "
              + f" {denoising_end} when using type float."
          )
      elif denoising_end is not None and denoising_value_valid(
          denoising_end
      ):
          discrete_timestep_cutoff = int(
              round(
                  self.scheduler.config.num_train_timesteps
                  - (
                      denoising_end
                      * self.scheduler.config.num_train_timesteps
                  )
              )
          )
          num_inference_steps = len(
              list(
                  filter(
                      lambda ts: ts >= discrete_timestep_cutoff,
                      timesteps,
                  )
              )
          )
          timesteps = timesteps[:num_inference_steps]
  
      with self.progress_bar(total=num_inference_steps) as progress_bar:
          for i, t in enumerate(timesteps):
              # expand the latents if we are doing classifier free guidance
              latent_model_input = (
                  torch.cat([latents] * 2)
                  if do_classifier_free_guidance
                  else latents
              )
  
              # concat latents, mask, masked_image_latents in the channel dimension
              latent_model_input = self.scheduler.scale_model_input(
                  latent_model_input, t
              )
  
              if num_channels_unet == 9:
                  latent_model_input = torch.cat(
                      [latent_model_input, mask, masked_image_latents],
                      dim=1,
                  )
  
              # predict the noise residual
              added_cond_kwargs = {
                  "text_embeds": add_text_embeds,
                  "time_ids": add_time_ids,
              }
              noise_pred = self.unet(
                  latent_model_input,
                  t,
                  encoder_hidden_states=prompt_embeds,
                  cross_attention_kwargs=cross_attention_kwargs,
                  added_cond_kwargs=added_cond_kwargs,
                  return_dict=False,
              )[0]
  
              # perform guidance
              if do_classifier_free_guidance:
                  noise_pred_uncond, noise_pred_text = noise_pred.chunk(
                      2
                  )
                  noise_pred = noise_pred_uncond + guidance_scale * (
                      noise_pred_text - noise_pred_uncond
                  )
  
              if do_classifier_free_guidance and guidance_rescale > 0.0:
                  # Based on 3.4. in https://arxiv.org/pdf/2305.08891.pdf
                  noise_pred = rescale_noise_cfg(
                      noise_pred,
                      noise_pred_text,
                      guidance_rescale=guidance_rescale,
                  )
  
              # compute the previous noisy sample x_t -> x_t-1
              latents = self.scheduler.step(
                  noise_pred,
                  t,
                  latents,
                  **extra_step_kwargs,
                  return_dict=False,
              )[0]
  
              if num_channels_unet == 4:
                  init_latents_proper = image_latents
                  if do_classifier_free_guidance:
                      init_mask, _ = mask.chunk(2)
                  else:
                      init_mask = mask
  
                  if i < len(timesteps) - 1:
                      noise_timestep = timesteps[i + 1]
                      init_latents_proper = self.scheduler.add_noise(
                          init_latents_proper,
                          noise,
                          torch.tensor([noise_timestep]),
                      )
  
                  latents = (
                      1 - init_mask
                  ) * init_latents_proper + init_mask * latents
  
              # call the callback, if provided
              if i == len(timesteps) - 1 or (
                  (i + 1) > num_warmup_steps
                  and (i + 1) % self.scheduler.order == 0
              ):
                  progress_bar.update()
                  if callback is not None and i % callback_steps == 0:
                      step_idx = i // getattr(
                          self.scheduler, "order", 1
                      )
                      callback(step_idx, t, latents)
  
              if XLA_AVAILABLE:
                  xm.mark_step()
  
      if not output_type == "latent":
          # make sure the VAE is in float32 mode, as it overflows in float16
          needs_upcasting = (
              self.vae.dtype == torch.float16
              and self.vae.config.force_upcast
          )
  
          if needs_upcasting:
              self.upcast_vae()
              latents = latents.to(
                  next(
                      iter(self.vae.post_quant_conv.parameters())
                  ).dtype
              )
  
          image = self.vae.decode(
              latents / self.vae.config.scaling_factor,
              return_dict=False,
          )[0]
  
          # cast back to fp16 if needed
          if needs_upcasting:
              self.vae.to(dtype=torch.float16)
      else:
          return StableDiffusionXLPipelineOutput(images=latents)
  
      # apply watermark if available
      if self.watermark is not None:
          image = self.watermark.apply_watermark(image)
  
      image = self.image_processor.postprocess(
          image, output_type=output_type
      )
  
      # Offload all models
      self.maybe_free_model_hooks()
  
      if not return_dict:
          return (image,)
  
      return StableDiffusionXLPipelineOutput(images=image)
  
imports: import inspect
  from typing import Any, Callable, Dict, List, Optional, Tuple, Union
  import numpy
  import PIL.Image
  import torch
  from transformers import CLIPTextModel, CLIPTextModelWithProjection, CLIPTokenizer
  from diffusers.image_processor import PipelineImageInput, VaeImageProcessor
  from diffusers.loaders import FromSingleFileMixin, StableDiffusionXLLoraLoaderMixin, TextualInversionLoaderMixin
  from diffusers.models import AutoencoderKL, UNet2DConditionModel
  from diffusers.models.attention_processor import AttnProcessor2_0, LoRAAttnProcessor2_0, LoRAXFormersAttnProcessor, XFormersAttnProcessor
  from diffusers.models.lora import adjust_lora_scale_text_encoder
  from diffusers.schedulers import KarrasDiffusionSchedulers
  from diffusers.utils import USE_PEFT_BACKEND, deprecate, is_invisible_watermark_available, is_torch_xla_available, logging, replace_example_docstring, scale_lora_layers, unscale_lora_layers
  from diffusers.utils.torch_utils import randn_tensor
  from diffusers.pipelines.pipeline_utils import DiffusionPipeline
  from diffusers.pipelines.stable_diffusion_xl.pipeline_output import StableDiffusionXLPipelineOutput
  from llmga.diffusers.my_utils.util import get_unweighted_text_embeddings_SDXL1, get_unweighted_text_embeddings_SDXL2, get_text_index
label_desc: Use image node_id 17 for calling the StableDiffusionXLInpaintPipeline obj's underlying member method code with example usage: obj.__call__(prompt, prompt_2, image, mask_image, masked_image_latents, height, width, strength, num_inference_steps, denoising_start, denoising_end, guidance_scale, negative_prompt, negative_prompt_2, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, pooled_prompt_embeds, negative_pooled_prompt_embeds, output_type, return_dict, callback, callback_steps, cross_attention_kwargs, guidance_rescale, original_size, crops_coords_top_left, target_size, negative_original_size, negative_crops_coords_top_left, negative_target_size, aesthetic_score, negative_aesthetic_score, clip_skip) and returns: StableDiffusionXLPipelineOutput, unknown, StableDiffusionXLPipelineOutput, image