class_docstr: Pipeline for text-to-image generation using Stable Diffusion.
  
  This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods
  implemented for all pipelines (downloading, saving, running on a particular device, etc.).
  
  The pipeline also inherits the following loading methods:
      - [`~loaders.TextualInversionLoaderMixin.load_textual_inversion`] for loading textual inversion embeddings
      - [`~loaders.IPAdapterMixin.load_ip_adapter`] for loading IP Adapters
  
  Args:
      vae ([`AutoencoderKL`]):
          Variational Auto-Encoder (VAE) model to encode and decode images to and from latent representations.
      text_encoder ([`~transformers.CLIPTextModel`]):
          Frozen text-encoder ([clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)).
      tokenizer ([`~transformers.CLIPTokenizer`]):
          A `CLIPTokenizer` to tokenize text.
      unet ([`UNet2DConditionModel`]):
          A `UNet2DConditionModel` to denoise the encoded image latents.
      scheduler ([`SchedulerMixin`]):
          A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of
          [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`].
      safety_checker ([`StableDiffusionSafetyChecker`]):
          Classification module that estimates whether generated images could be considered offensive or harmful.
          Please refer to the [model card](https://huggingface.co/runwayml/stable-diffusion-v1-5) for more details
          about a model's potential harms.
      feature_extractor ([`~transformers.CLIPImageProcessor`]):
          A `CLIPImageProcessor` to extract features from generated images; used as inputs to the `safety_checker`.
docstr: The call function to the pipeline for generation.
  
  Args:
      prompt (`str` or `List[str]`, *optional*):
          The prompt or prompts to guide image generation. If not defined, you need to pass `prompt_embeds`.
      height (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`):
          The height in pixels of the generated image.
      width (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`):
          The width in pixels of the generated image.
      num_inference_steps (`int`, *optional*, defaults to 50):
          The number of denoising steps. More denoising steps usually lead to a higher quality image at the
          expense of slower inference.
      guidance_scale (`float`, *optional*, defaults to 7.5):
          A higher guidance scale value encourages the model to generate images closely linked to the text
          `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`.
      sag_scale (`float`, *optional*, defaults to 0.75):
          Chosen between [0, 1.0] for better quality.
      negative_prompt (`str` or `List[str]`, *optional*):
          The prompt or prompts to guide what to not include in image generation. If not defined, you need to
          pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`).
      num_images_per_prompt (`int`, *optional*, defaults to 1):
          The number of images to generate per prompt.
      eta (`float`, *optional*, defaults to 0.0):
          Corresponds to parameter eta (η) from the [DDIM](https://arxiv.org/abs/2010.02502) paper. Only applies
          to the [`~schedulers.DDIMScheduler`], and is ignored in other schedulers.
      generator (`torch.Generator` or `List[torch.Generator]`, *optional*):
          A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make
          generation deterministic.
      latents (`torch.FloatTensor`, *optional*):
          Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image
          generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
          tensor is generated by sampling using the supplied random `generator`.
      prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not
          provided, text embeddings are generated from the `prompt` input argument.
      negative_prompt_embeds (`torch.FloatTensor`, *optional*):
          Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If
          not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument.
      ip_adapter_image: (`PipelineImageInput`, *optional*):
          Optional image input to work with IP Adapters.
      output_type (`str`, *optional*, defaults to `"pil"`):
          The output format of the generated image. Choose between `PIL.Image` or `np.array`.
      return_dict (`bool`, *optional*, defaults to `True`):
          Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
          plain tuple.
      callback (`Callable`, *optional*):
          A function that calls every `callback_steps` steps during inference. The function is called with the
          following arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`.
      callback_steps (`int`, *optional*, defaults to 1):
          The frequency at which the `callback` function is called. If not specified, the callback is called at
          every step.
      cross_attention_kwargs (`dict`, *optional*):
          A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in
          [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
      clip_skip (`int`, *optional*):
          Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that
          the output of the pre-final layer will be used for computing the prompt embeddings.
  Examples:
  
  Returns:
      [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] or `tuple`:
          If `return_dict` is `True`, [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] is returned,
          otherwise a `tuple` is returned where the first element is a list with the generated images and the
          second element is a list of `bool`s indicating whether the corresponding generated image contains
          "not-safe-for-work" (nsfw) content.
code: def __call__(
      self,
      prompt: Union[str, List[str]] = None,
      height: Optional[int] = None,
      width: Optional[int] = None,
      num_inference_steps: int = 50,
      guidance_scale: float = 7.5,
      sag_scale: float = 0.75,
      negative_prompt: Optional[Union[str, List[str]]] = None,
      num_images_per_prompt: Optional[int] = 1,
      eta: float = 0.0,
      generator: Optional[
          Union[torch.Generator, List[torch.Generator]]
      ] = None,
      latents: Optional[torch.FloatTensor] = None,
      prompt_embeds: Optional[torch.FloatTensor] = None,
      negative_prompt_embeds: Optional[torch.FloatTensor] = None,
      ip_adapter_image: Optional[PipelineImageInput] = None,
      output_type: Optional[str] = "pil",
      return_dict: bool = True,
      callback: Optional[
          Callable[[int, int, torch.FloatTensor], None]
      ] = None,
      callback_steps: Optional[int] = 1,
      cross_attention_kwargs: Optional[Dict[str, Any]] = None,
      clip_skip: Optional[int] = None,
  ):
      r"""
      The call function to the pipeline for generation.
  
      Args:
          prompt (`str` or `List[str]`, *optional*):
              The prompt or prompts to guide image generation. If not defined, you need to pass `prompt_embeds`.
          height (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`):
              The height in pixels of the generated image.
          width (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`):
              The width in pixels of the generated image.
          num_inference_steps (`int`, *optional*, defaults to 50):
              The number of denoising steps. More denoising steps usually lead to a higher quality image at the
              expense of slower inference.
          guidance_scale (`float`, *optional*, defaults to 7.5):
              A higher guidance scale value encourages the model to generate images closely linked to the text
              `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`.
          sag_scale (`float`, *optional*, defaults to 0.75):
              Chosen between [0, 1.0] for better quality.
          negative_prompt (`str` or `List[str]`, *optional*):
              The prompt or prompts to guide what to not include in image generation. If not defined, you need to
              pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`).
          num_images_per_prompt (`int`, *optional*, defaults to 1):
              The number of images to generate per prompt.
          eta (`float`, *optional*, defaults to 0.0):
              Corresponds to parameter eta (η) from the [DDIM](https://arxiv.org/abs/2010.02502) paper. Only applies
              to the [`~schedulers.DDIMScheduler`], and is ignored in other schedulers.
          generator (`torch.Generator` or `List[torch.Generator]`, *optional*):
              A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make
              generation deterministic.
          latents (`torch.FloatTensor`, *optional*):
              Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image
              generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
              tensor is generated by sampling using the supplied random `generator`.
          prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not
              provided, text embeddings are generated from the `prompt` input argument.
          negative_prompt_embeds (`torch.FloatTensor`, *optional*):
              Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If
              not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument.
          ip_adapter_image: (`PipelineImageInput`, *optional*):
              Optional image input to work with IP Adapters.
          output_type (`str`, *optional*, defaults to `"pil"`):
              The output format of the generated image. Choose between `PIL.Image` or `np.array`.
          return_dict (`bool`, *optional*, defaults to `True`):
              Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
              plain tuple.
          callback (`Callable`, *optional*):
              A function that calls every `callback_steps` steps during inference. The function is called with the
              following arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`.
          callback_steps (`int`, *optional*, defaults to 1):
              The frequency at which the `callback` function is called. If not specified, the callback is called at
              every step.
          cross_attention_kwargs (`dict`, *optional*):
              A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in
              [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py).
          clip_skip (`int`, *optional*):
              Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that
              the output of the pre-final layer will be used for computing the prompt embeddings.
      Examples:
  
      Returns:
          [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] or `tuple`:
              If `return_dict` is `True`, [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] is returned,
              otherwise a `tuple` is returned where the first element is a list with the generated images and the
              second element is a list of `bool`s indicating whether the corresponding generated image contains
              "not-safe-for-work" (nsfw) content.
      """
      # 0. Default height and width to unet
      height = (
          height or self.unet.config.sample_size * self.vae_scale_factor
      )
      width = (
          width or self.unet.config.sample_size * self.vae_scale_factor
      )
  
      # 1. Check inputs. Raise error if not correct
      self.check_inputs(
          prompt,
          height,
          width,
          callback_steps,
          negative_prompt,
          prompt_embeds,
          negative_prompt_embeds,
      )
  
      # 2. Define call parameters
      if prompt is not None and isinstance(prompt, str):
          batch_size = 1
      elif prompt is not None and isinstance(prompt, list):
          batch_size = len(prompt)
      else:
          batch_size = prompt_embeds.shape[0]
  
      device = self._execution_device
      # here `guidance_scale` is defined analog to the guidance weight `w` of equation (2)
      # of the Imagen paper: https://arxiv.org/pdf/2205.11487.pdf . `guidance_scale = 1`
      # corresponds to doing no classifier free guidance.
      do_classifier_free_guidance = guidance_scale > 1.0
      # and `sag_scale` is` `s` of equation (16)
      # of the self-attentnion guidance paper: https://arxiv.org/pdf/2210.00939.pdf
      # `sag_scale = 0` means no self-attention guidance
      do_self_attention_guidance = sag_scale > 0.0
  
      if ip_adapter_image is not None:
          output_hidden_state = (
              False
              if isinstance(self.unet.encoder_hid_proj, ImageProjection)
              else True
          )
          image_embeds, negative_image_embeds = self.encode_image(
              ip_adapter_image,
              device,
              num_images_per_prompt,
              output_hidden_state,
          )
          if do_classifier_free_guidance:
              image_embeds = torch.cat(
                  [negative_image_embeds, image_embeds]
              )
  
      # 3. Encode input prompt
      prompt_embeds, negative_prompt_embeds = self.encode_prompt(
          prompt,
          device,
          num_images_per_prompt,
          do_classifier_free_guidance,
          negative_prompt,
          prompt_embeds=prompt_embeds,
          negative_prompt_embeds=negative_prompt_embeds,
          clip_skip=clip_skip,
      )
      # For classifier free guidance, we need to do two forward passes.
      # Here we concatenate the unconditional and text embeddings into a single batch
      # to avoid doing two forward passes
      if do_classifier_free_guidance:
          prompt_embeds = torch.cat(
              [negative_prompt_embeds, prompt_embeds]
          )
  
      # 4. Prepare timesteps
      self.scheduler.set_timesteps(num_inference_steps, device=device)
      timesteps = self.scheduler.timesteps
  
      if timesteps.dtype not in [torch.int16, torch.int32, torch.int64]:
          raise ValueError(
              f"{self.__class__.__name__} does not support using a scheduler of type {self.scheduler.__class__.__name__}. Please make sure to use one of 'DDIMScheduler, PNDMScheduler, DDPMScheduler, DEISMultistepScheduler, UniPCMultistepScheduler, DPMSolverMultistepScheduler, DPMSolverSinlgestepScheduler'."
          )
  
      # 5. Prepare latent variables
      num_channels_latents = self.unet.config.in_channels
      latents = self.prepare_latents(
          batch_size * num_images_per_prompt,
          num_channels_latents,
          height,
          width,
          prompt_embeds.dtype,
          device,
          generator,
          latents,
      )
  
      # 6. Prepare extra step kwargs. TODO: Logic should ideally just be moved out of the pipeline
      extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
  
      # 6.1 Add image embeds for IP-Adapter
      added_cond_kwargs = (
          {"image_embeds": image_embeds}
          if ip_adapter_image is not None
          else None
      )
      added_uncond_kwargs = (
          {"image_embeds": negative_image_embeds}
          if ip_adapter_image is not None
          else None
      )
  
      # 7. Denoising loop
      store_processor = CrossAttnStoreProcessor()
      self.unet.mid_block.attentions[0].transformer_blocks[
          0
      ].attn1.processor = store_processor
      num_warmup_steps = (
          len(timesteps) - num_inference_steps * self.scheduler.order
      )
  
      map_size = None
  
      def get_map_size(module, input, output):
          nonlocal map_size
          map_size = output[0].shape[-2:]
  
      with self.unet.mid_block.attentions[0].register_forward_hook(
          get_map_size
      ):
          with self.progress_bar(
              total=num_inference_steps
          ) as progress_bar:
              for i, t in enumerate(timesteps):
                  # expand the latents if we are doing classifier free guidance
                  latent_model_input = (
                      torch.cat([latents] * 2)
                      if do_classifier_free_guidance
                      else latents
                  )
                  latent_model_input = self.scheduler.scale_model_input(
                      latent_model_input, t
                  )
  
                  # predict the noise residual
  
                  noise_pred = self.unet(
                      latent_model_input,
                      t,
                      encoder_hidden_states=prompt_embeds,
                      cross_attention_kwargs=cross_attention_kwargs,
                      added_cond_kwargs=added_cond_kwargs,
                  ).sample
  
                  # perform guidance
                  if do_classifier_free_guidance:
                      (
                          noise_pred_uncond,
                          noise_pred_text,
                      ) = noise_pred.chunk(2)
                      noise_pred = (
                          noise_pred_uncond
                          + guidance_scale
                          * (noise_pred_text - noise_pred_uncond)
                      )
  
                  # perform self-attention guidance with the stored self-attentnion map
                  if do_self_attention_guidance:
                      # classifier-free guidance produces two chunks of attention map
                      # and we only use unconditional one according to equation (25)
                      # in https://arxiv.org/pdf/2210.00939.pdf
                      if do_classifier_free_guidance:
                          # DDIM-like prediction of x0
                          pred_x0 = self.pred_x0(
                              latents, noise_pred_uncond, t
                          )
                          # get the stored attention maps
                          (
                              uncond_attn,
                              cond_attn,
                          ) = store_processor.attention_probs.chunk(2)
                          # self-attention-based degrading of latents
                          degraded_latents = self.sag_masking(
                              pred_x0,
                              uncond_attn,
                              map_size,
                              t,
                              self.pred_epsilon(
                                  latents, noise_pred_uncond, t
                              ),
                          )
                          uncond_emb, _ = prompt_embeds.chunk(2)
                          # forward and give guidance
                          degraded_pred = self.unet(
                              degraded_latents,
                              t,
                              encoder_hidden_states=uncond_emb,
                              added_cond_kwargs=added_uncond_kwargs,
                          ).sample
                          noise_pred += sag_scale * (
                              noise_pred_uncond - degraded_pred
                          )
                      else:
                          # DDIM-like prediction of x0
                          pred_x0 = self.pred_x0(latents, noise_pred, t)
                          # get the stored attention maps
                          cond_attn = store_processor.attention_probs
                          # self-attention-based degrading of latents
                          degraded_latents = self.sag_masking(
                              pred_x0,
                              cond_attn,
                              map_size,
                              t,
                              self.pred_epsilon(latents, noise_pred, t),
                          )
                          # forward and give guidance
                          degraded_pred = self.unet(
                              degraded_latents,
                              t,
                              encoder_hidden_states=prompt_embeds,
                              added_cond_kwargs=added_cond_kwargs,
                          ).sample
                          noise_pred += sag_scale * (
                              noise_pred - degraded_pred
                          )
  
                  # compute the previous noisy sample x_t -> x_t-1
                  latents = self.scheduler.step(
                      noise_pred, t, latents, **extra_step_kwargs
                  ).prev_sample
  
                  # call the callback, if provided
                  if i == len(timesteps) - 1 or (
                      (i + 1) > num_warmup_steps
                      and (i + 1) % self.scheduler.order == 0
                  ):
                      progress_bar.update()
                      if (
                          callback is not None
                          and i % callback_steps == 0
                      ):
                          step_idx = i // getattr(
                              self.scheduler, "order", 1
                          )
                          callback(step_idx, t, latents)
  
      if not output_type == "latent":
          image = self.vae.decode(
              latents / self.vae.config.scaling_factor,
              return_dict=False,
          )[0]
          image, has_nsfw_concept = self.run_safety_checker(
              image, device, prompt_embeds.dtype
          )
      else:
          image = latents
          has_nsfw_concept = None
  
      if has_nsfw_concept is None:
          do_denormalize = [True] * image.shape[0]
      else:
          do_denormalize = [
              not has_nsfw for has_nsfw in has_nsfw_concept
          ]
  
      image = self.image_processor.postprocess(
          image, output_type=output_type, do_denormalize=do_denormalize
      )
  
      self.maybe_free_model_hooks()
  
      if not return_dict:
          return (image, has_nsfw_concept)
  
      return StableDiffusionPipelineOutput(
          images=image, nsfw_content_detected=has_nsfw_concept
      )
  
imports: import inspect
  from typing import Any, Callable, Dict, List, Optional, Union
  import torch
  import torch.nn.functional
  from transformers import CLIPImageProcessor, CLIPTextModel, CLIPTokenizer, CLIPVisionModelWithProjection
  from ...image_processor import PipelineImageInput, VaeImageProcessor
  from ...loaders import IPAdapterMixin, LoraLoaderMixin, TextualInversionLoaderMixin
  from ...models import AutoencoderKL, ImageProjection, UNet2DConditionModel
  from ...models.lora import adjust_lora_scale_text_encoder
  from ...schedulers import KarrasDiffusionSchedulers
  from ...utils import USE_PEFT_BACKEND, deprecate, logging, replace_example_docstring, scale_lora_layers, unscale_lora_layers
  from ...utils.torch_utils import randn_tensor
  from ..pipeline_utils import DiffusionPipeline
  from ..stable_diffusion import StableDiffusionPipelineOutput
  from ..stable_diffusion.safety_checker import StableDiffusionSafetyChecker
label_desc: Use image node_id 12 for calling the StableDiffusionSAGPipeline obj's underlying member method code with example usage: obj.__call__(prompt, height, width, num_inference_steps, guidance_scale, sag_scale, negative_prompt, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, ip_adapter_image, output_type, return_dict, callback, callback_steps, cross_attention_kwargs, clip_skip) and returns: StableDiffusionPipelineOutput, image,  has_nsfw_concept