class_docstr: Pipeline for text-to-image generation using Stable Diffusion without tokens length limit, and support parsing
  weighting in prompt.
  
  This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the
  library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.)
class_docstr_tok: ["Pipeline","for","text-to-image","generation","using","Stable","Diffusion","without","tokens","length","limit",",","and","support","parsing","weighting","in","prompt",".","This","model","inherits","from","[","`","DiffusionPipeline","`","]",".","Check","the","superclass","documentation","for","the","generic","methods","the","library","implements","for","all","the","pipelines","(","such","as","downloading","or","saving",",","running","on","a","particular","device",",","etc",".",")"]
docstr: Function invoked when calling the pipeline for generation.
  
  Args:
      prompt (`str` or `List[str]`):
          The prompt or prompts to guide the image generation.
      negative_prompt (`str` or `List[str]`, *optional*):
          The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored
          if `guidance_scale` is less than `1`).
      image (`np.ndarray` or `PIL.Image.Image`):
          `Image`, or tensor representing an image batch, that will be used as the starting point for the
          process.
      mask_image (`np.ndarray` or `PIL.Image.Image`):
          `Image`, or tensor representing an image batch, to mask `image`. White pixels in the mask will be
          replaced by noise and therefore repainted, while black pixels will be preserved. If `mask_image` is a
          PIL image, it will be converted to a single channel (luminance) before use. If it's a tensor, it should
          contain one color channel (L) instead of 3, so the expected shape would be `(B, H, W, 1)`.
      height (`int`, *optional*, defaults to 512):
          The height in pixels of the generated image.
      width (`int`, *optional*, defaults to 512):
          The width in pixels of the generated image.
      num_inference_steps (`int`, *optional*, defaults to 50):
          The number of denoising steps. More denoising steps usually lead to a higher quality image at the
          expense of slower inference.
      guidance_scale (`float`, *optional*, defaults to 7.5):
          Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598).
          `guidance_scale` is defined as `w` of equation 2. of [Imagen
          Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale >
          1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`,
          usually at the expense of lower image quality.
      strength (`float`, *optional*, defaults to 0.8):
          Conceptually, indicates how much to transform the reference `image`. Must be between 0 and 1.
          `image` will be used as a starting point, adding more noise to it the larger the `strength`. The
          number of denoising steps depends on the amount of noise initially added. When `strength` is 1, added
          noise will be maximum and the denoising process will run for the full number of iterations specified in
          `num_inference_steps`. A value of 1, therefore, essentially ignores `image`.
      num_images_per_prompt (`int`, *optional*, defaults to 1):
          The number of images to generate per prompt.
      eta (`float`, *optional*, defaults to 0.0):
          Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to
          [`schedulers.DDIMScheduler`], will be ignored for others.
      generator (`torch.Generator`, *optional*):
          A [torch generator](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation
          deterministic.
      latents (`np.ndarray`, *optional*):
          Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image
          generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
          tensor will ge generated by sampling using the supplied random `generator`.
      max_embeddings_multiples (`int`, *optional*, defaults to `3`):
          The max multiple length of prompt embeddings compared to the max output length of text encoder.
      output_type (`str`, *optional*, defaults to `"pil"`):
          The output format of the generate image. Choose between
          [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`.
      return_dict (`bool`, *optional*, defaults to `True`):
          Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
          plain tuple.
      callback (`Callable`, *optional*):
          A function that will be called every `callback_steps` steps during inference. The function will be
          called with the following arguments: `callback(step: int, timestep: int, latents: np.ndarray)`.
      is_cancelled_callback (`Callable`, *optional*):
          A function that will be called every `callback_steps` steps during inference. If the function returns
          `True`, the inference will be cancelled.
      callback_steps (`int`, *optional*, defaults to 1):
          The frequency at which the `callback` function will be called. If not specified, the callback will be
          called at every step.
  
  Returns:
      `None` if cancelled by `is_cancelled_callback`,
      [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] or `tuple`:
      [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] if `return_dict` is True, otherwise a `tuple.
      When returning a tuple, the first element is a list with the generated images, and the second element is a
      list of `bool`s denoting whether the corresponding generated image likely represents "not-safe-for-work"
      (nsfw) content, according to the `safety_checker`.
docstr_tok: ["Function","invoked","when","calling","the","pipeline","for","generation",".","Args",":","prompt","(","`","str","`","or","`","List","[","str","]","`",")",":","The","prompt","or","prompts","to","guide","the","image","generation",".","negative_prompt","(","`","str","`","or","`","List","[","str","]","`",",","*","optional","*",")",":","The","prompt","or","prompts","not","to","guide","the","image","generation",".","Ignored","when","not","using","guidance","(","i.e.",",","ignored","if","`","guidance_scale","`","is","less","than","`","1","`",")",".","image","(","`","np.ndarray","`","or","`","PIL.Image.Image","`",")",":","`","Image","`",",","or","tensor","representing","an","image","batch",",","that","will","be","used","as","the","starting","point","for","the","process",".","mask_image","(","`","np.ndarray","`","or","`","PIL.Image.Image","`",")",":","`","Image","`",",","or","tensor","representing","an","image","batch",",","to","mask","`","image","`",".","White","pixels","in","the","mask","will","be","replaced","by","noise","and","therefore","repainted",",","while","black","pixels","will","be","preserved",".","If","`","mask_image","`","is","a","PIL","image",",","it","will","be","converted","to","a","single","channel","(","luminance",")","before","use",".","If","it","'s","a","tensor",",","it","should","contain","one","color","channel","(","L",")","instead","of","3",",","so","the","expected","shape","would","be","`","(","B",",","H",",","W",",","1",")","`",".","height","(","`","int","`",",","*","optional","*",",","defaults","to","512",")",":","The","height","in","pixels","of","the","generated","image",".","width","(","`","int","`",",","*","optional","*",",","defaults","to","512",")",":","The","width","in","pixels","of","the","generated","image",".","num_inference_steps","(","`","int","`",",","*","optional","*",",","defaults","to","50",")",":","The","number","of","denoising","steps",".","More","denoising","steps","usually","lead","to","a","higher","quality","image","at","the","expense","of","slower","inference",".","guidance_scale","(","`","float","`",",","*","optional","*",",","defaults","to","7.5",")",":","Guidance","scale","as","defined","in","[","Classifier-Free","Diffusion","Guidance","]","(","https",":","\/\/arxiv.org\/abs\/2207.12598",")",".","`","guidance_scale","`","is","defined","as","`","w","`","of","equation","2.","of","[","Imagen","Paper","]","(","https",":","\/\/arxiv.org\/pdf\/2205.11487.pdf",")",".","Guidance","scale","is","enabled","by","setting","`","guidance_scale",">","1","`",".","Higher","guidance","scale","encourages","to","generate","images","that","are","closely","linked","to","the","text","`","prompt","`",",","usually","at","the","expense","of","lower","image","quality",".","strength","(","`","float","`",",","*","optional","*",",","defaults","to","0.8",")",":","Conceptually",",","indicates","how","much","to","transform","the","reference","`","image","`",".","Must","be","between","0","and","1",".","`","image","`","will","be","used","as","a","starting","point",",","adding","more","noise","to","it","the","larger","the","`","strength","`",".","The","number","of","denoising","steps","depends","on","the","amount","of","noise","initially","added",".","When","`","strength","`","is","1",",","added","noise","will","be","maximum","and","the","denoising","process","will","run","for","the","full","number","of","iterations","specified","in","`","num_inference_steps","`",".","A","value","of","1",",","therefore",",","essentially","ignores","`","image","`",".","num_images_per_prompt","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","number","of","images","to","generate","per","prompt",".","eta","(","`","float","`",",","*","optional","*",",","defaults","to","0.0",")",":","Corresponds","to","parameter","eta","(","\u03b7",")","in","the","DDIM","paper",":","https",":","\/\/arxiv.org\/abs\/2010.02502",".","Only","applies","to","[","`","schedulers.DDIMScheduler","`","]",",","will","be","ignored","for","others",".","generator","(","`","torch.Generator","`",",","*","optional","*",")",":","A","[","torch","generator","]","(","https",":","\/\/pytorch.org\/docs\/stable\/generated\/torch.Generator.html",")","to","make","generation","deterministic",".","latents","(","`","np.ndarray","`",",","*","optional","*",")",":","Pre-generated","noisy","latents",",","sampled","from","a","Gaussian","distribution",",","to","be","used","as","inputs","for","image","generation",".","Can","be","used","to","tweak","the","same","generation","with","different","prompts",".","If","not","provided",",","a","latents","tensor","will","ge","generated","by","sampling","using","the","supplied","random","`","generator","`",".","max_embeddings_multiples","(","`","int","`",",","*","optional","*",",","defaults","to","`","3","`",")",":","The","max","multiple","length","of","prompt","embeddings","compared","to","the","max","output","length","of","text","encoder",".","output_type","(","`","str","`",",","*","optional","*",",","defaults","to","`","``","pil","''","`",")",":","The","output","format","of","the","generate","image",".","Choose","between","[","PIL","]","(","https",":","\/\/pillow.readthedocs.io\/en\/stable\/",")",":","`","PIL.Image.Image","`","or","`","np.array","`",".","return_dict","(","`","bool","`",",","*","optional","*",",","defaults","to","`","True","`",")",":","Whether","or","not","to","return","a","[","`","~pipelines.stable_diffusion.StableDiffusionPipelineOutput","`","]","instead","of","a","plain","tuple",".","callback","(","`","Callable","`",",","*","optional","*",")",":","A","function","that","will","be","called","every","`","callback_steps","`","steps","during","inference",".","The","function","will","be","called","with","the","following","arguments",":","`","callback","(","step",":","int",",","timestep",":","int",",","latents",":","np.ndarray",")","`",".","is_cancelled_callback","(","`","Callable","`",",","*","optional","*",")",":","A","function","that","will","be","called","every","`","callback_steps","`","steps","during","inference",".","If","the","function","returns","`","True","`",",","the","inference","will","be","cancelled",".","callback_steps","(","`","int","`",",","*","optional","*",",","defaults","to","1",")",":","The","frequency","at","which","the","`","callback","`","function","will","be","called",".","If","not","specified",",","the","callback","will","be","called","at","every","step",".","Returns",":","`","None","`","if","cancelled","by","`","is_cancelled_callback","`",",","[","`","~pipelines.stable_diffusion.StableDiffusionPipelineOutput","`","]","or","`","tuple","`",":","[","`","~pipelines.stable_diffusion.StableDiffusionPipelineOutput","`","]","if","`","return_dict","`","is","True",",","otherwise","a","`","tuple",".","When","returning","a","tuple",",","the","first","element","is","a","list","with","the","generated","images",",","and","the","second","element","is","a","list","of","`","bool","`","s","denoting","whether","the","corresponding","generated","image","likely","represents","``","not-safe-for-work","''","(","nsfw",")","content",",","according","to","the","`","safety_checker","`","."]
code: def __call__(
      self,
      prompt: Union[str, List[str]],
      negative_prompt: Optional[Union[str, List[str]]] = None,
      image: Union[np.ndarray, PIL.Image.Image] = None,
      mask_image: Union[np.ndarray, PIL.Image.Image] = None,
      height: int = 512,
      width: int = 512,
      num_inference_steps: int = 50,
      guidance_scale: float = 7.5,
      strength: float = 0.8,
      num_images_per_prompt: Optional[int] = 1,
      eta: float = 0.0,
      generator: Optional[torch.Generator] = None,
      latents: Optional[np.ndarray] = None,
      max_embeddings_multiples: Optional[int] = 3,
      output_type: Optional[str] = "pil",
      return_dict: bool = True,
      callback: Optional[Callable[[int, int, np.ndarray], None]] = None,
      is_cancelled_callback: Optional[Callable[[], bool]] = None,
      callback_steps: int = 1,
      **kwargs,
  ):
      r"""
      Function invoked when calling the pipeline for generation.
  
      Args:
          prompt (`str` or `List[str]`):
              The prompt or prompts to guide the image generation.
          negative_prompt (`str` or `List[str]`, *optional*):
              The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored
              if `guidance_scale` is less than `1`).
          image (`np.ndarray` or `PIL.Image.Image`):
              `Image`, or tensor representing an image batch, that will be used as the starting point for the
              process.
          mask_image (`np.ndarray` or `PIL.Image.Image`):
              `Image`, or tensor representing an image batch, to mask `image`. White pixels in the mask will be
              replaced by noise and therefore repainted, while black pixels will be preserved. If `mask_image` is a
              PIL image, it will be converted to a single channel (luminance) before use. If it's a tensor, it should
              contain one color channel (L) instead of 3, so the expected shape would be `(B, H, W, 1)`.
          height (`int`, *optional*, defaults to 512):
              The height in pixels of the generated image.
          width (`int`, *optional*, defaults to 512):
              The width in pixels of the generated image.
          num_inference_steps (`int`, *optional*, defaults to 50):
              The number of denoising steps. More denoising steps usually lead to a higher quality image at the
              expense of slower inference.
          guidance_scale (`float`, *optional*, defaults to 7.5):
              Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598).
              `guidance_scale` is defined as `w` of equation 2. of [Imagen
              Paper](https://arxiv.org/pdf/2205.11487.pdf). Guidance scale is enabled by setting `guidance_scale >
              1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`,
              usually at the expense of lower image quality.
          strength (`float`, *optional*, defaults to 0.8):
              Conceptually, indicates how much to transform the reference `image`. Must be between 0 and 1.
              `image` will be used as a starting point, adding more noise to it the larger the `strength`. The
              number of denoising steps depends on the amount of noise initially added. When `strength` is 1, added
              noise will be maximum and the denoising process will run for the full number of iterations specified in
              `num_inference_steps`. A value of 1, therefore, essentially ignores `image`.
          num_images_per_prompt (`int`, *optional*, defaults to 1):
              The number of images to generate per prompt.
          eta (`float`, *optional*, defaults to 0.0):
              Corresponds to parameter eta (η) in the DDIM paper: https://arxiv.org/abs/2010.02502. Only applies to
              [`schedulers.DDIMScheduler`], will be ignored for others.
          generator (`torch.Generator`, *optional*):
              A [torch generator](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation
              deterministic.
          latents (`np.ndarray`, *optional*):
              Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image
              generation. Can be used to tweak the same generation with different prompts. If not provided, a latents
              tensor will ge generated by sampling using the supplied random `generator`.
          max_embeddings_multiples (`int`, *optional*, defaults to `3`):
              The max multiple length of prompt embeddings compared to the max output length of text encoder.
          output_type (`str`, *optional*, defaults to `"pil"`):
              The output format of the generate image. Choose between
              [PIL](https://pillow.readthedocs.io/en/stable/): `PIL.Image.Image` or `np.array`.
          return_dict (`bool`, *optional*, defaults to `True`):
              Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a
              plain tuple.
          callback (`Callable`, *optional*):
              A function that will be called every `callback_steps` steps during inference. The function will be
              called with the following arguments: `callback(step: int, timestep: int, latents: np.ndarray)`.
          is_cancelled_callback (`Callable`, *optional*):
              A function that will be called every `callback_steps` steps during inference. If the function returns
              `True`, the inference will be cancelled.
          callback_steps (`int`, *optional*, defaults to 1):
              The frequency at which the `callback` function will be called. If not specified, the callback will be
              called at every step.
  
      Returns:
          `None` if cancelled by `is_cancelled_callback`,
          [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] or `tuple`:
          [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] if `return_dict` is True, otherwise a `tuple.
          When returning a tuple, the first element is a list with the generated images, and the second element is a
          list of `bool`s denoting whether the corresponding generated image likely represents "not-safe-for-work"
          (nsfw) content, according to the `safety_checker`.
      """
      # 0. Default height and width to unet
      height = (
          height or self.unet.config.sample_size * self.vae_scale_factor
      )
      width = (
          width or self.unet.config.sample_size * self.vae_scale_factor
      )
  
      # 1. Check inputs. Raise error if not correct
      self.check_inputs(prompt, height, width, strength, callback_steps)
  
      # 2. Define call parameters
      batch_size = 1 if isinstance(prompt, str) else len(prompt)
      # here `guidance_scale` is defined analog to the guidance weight `w` of equation (2)
      # of the Imagen paper: https://arxiv.org/pdf/2205.11487.pdf . `guidance_scale = 1`
      # corresponds to doing no classifier free guidance.
      do_classifier_free_guidance = guidance_scale > 1.0
  
      # 3. Encode input prompt
      text_embeddings = self._encode_prompt(
          prompt,
          num_images_per_prompt,
          do_classifier_free_guidance,
          negative_prompt,
          max_embeddings_multiples,
      )
      dtype = text_embeddings.dtype
  
      # 4. Preprocess image and mask
      if isinstance(image, PIL.Image.Image):
          image = preprocess_image(image)
      if image is not None:
          image = image.astype(dtype)
      if isinstance(mask_image, PIL.Image.Image):
          mask_image = preprocess_mask(
              mask_image, self.vae_scale_factor
          )
      if mask_image is not None:
          mask = mask_image.astype(dtype)
          mask = np.concatenate(
              [mask] * batch_size * num_images_per_prompt
          )
      else:
          mask = None
  
      # 5. set timesteps
      self.scheduler.set_timesteps(num_inference_steps)
      timestep_dtype = next(
          (
              input.type
              for input in self.unet.model.get_inputs()
              if input.name == "timestep"
          ),
          "tensor(float)",
      )
      timestep_dtype = ORT_TO_NP_TYPE[timestep_dtype]
      timesteps, num_inference_steps = self.get_timesteps(
          num_inference_steps, strength, image is None
      )
      latent_timestep = timesteps[:1].repeat(
          batch_size * num_images_per_prompt
      )
  
      # 6. Prepare latent variables
      latents, init_latents_orig, noise = self.prepare_latents(
          image,
          latent_timestep,
          batch_size * num_images_per_prompt,
          height,
          width,
          dtype,
          generator,
          latents,
      )
  
      # 7. Prepare extra step kwargs. TODO: Logic should ideally just be moved out of the pipeline
      extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
  
      # 8. Denoising loop
      for i, t in enumerate(self.progress_bar(timesteps)):
          # expand the latents if we are doing classifier free guidance
          latent_model_input = (
              np.concatenate([latents] * 2)
              if do_classifier_free_guidance
              else latents
          )
          latent_model_input = self.scheduler.scale_model_input(
              torch.from_numpy(latent_model_input), t
          )
          latent_model_input = latent_model_input.numpy()
  
          # predict the noise residual
          noise_pred = self.unet(
              sample=latent_model_input,
              timestep=np.array([t], dtype=timestep_dtype),
              encoder_hidden_states=text_embeddings,
          )
          noise_pred = noise_pred[0]
  
          # perform guidance
          if do_classifier_free_guidance:
              noise_pred_uncond, noise_pred_text = np.split(
                  noise_pred, 2
              )
              noise_pred = noise_pred_uncond + guidance_scale * (
                  noise_pred_text - noise_pred_uncond
              )
  
          # compute the previous noisy sample x_t -> x_t-1
          scheduler_output = self.scheduler.step(
              torch.from_numpy(noise_pred),
              t,
              torch.from_numpy(latents),
              **extra_step_kwargs,
          )
          latents = scheduler_output.prev_sample.numpy()
  
          if mask is not None:
              # masking
              init_latents_proper = self.scheduler.add_noise(
                  torch.from_numpy(init_latents_orig),
                  torch.from_numpy(noise),
                  t,
              ).numpy()
              latents = (init_latents_proper * mask) + (
                  latents * (1 - mask)
              )
  
          # call the callback, if provided
          if i % callback_steps == 0:
              if callback is not None:
                  step_idx = i // getattr(self.scheduler, "order", 1)
                  callback(step_idx, t, latents)
              if (
                  is_cancelled_callback is not None
                  and is_cancelled_callback()
              ):
                  return None
  
      # 9. Post-processing
      image = self.decode_latents(latents)
  
      # 10. Run safety checker
      image, has_nsfw_concept = self.run_safety_checker(image)
  
      # 11. Convert to PIL
      if output_type == "pil":
          image = self.numpy_to_pil(image)
  
      if not return_dict:
          return image, has_nsfw_concept
  
      return StableDiffusionPipelineOutput(
          images=image, nsfw_content_detected=has_nsfw_concept
      )
  
imports: import inspect
  import re
  from typing import Callable, List, Optional, Union
  import numpy
  import PIL.Image
  import torch
  from packaging import version
  from transformers import CLIPImageProcessor, CLIPTokenizer
  import diffusers
  from diffusers import OnnxRuntimeModel, OnnxStableDiffusionPipeline, SchedulerMixin
  from diffusers.pipelines.stable_diffusion import StableDiffusionPipelineOutput
  from diffusers.utils import logging
label_desc: Use image node_id 9 for calling the OnnxStableDiffusionLongPromptWeightingPipeline obj's underlying member method code with example usage: obj.__call__(prompt, negative_prompt, image, mask_image, height, width, num_inference_steps, guidance_scale, strength, num_images_per_prompt, eta, generator, latents, max_embeddings_multiples, output_type, return_dict, callback, is_cancelled_callback, callback_steps) and returns: StableDiffusionPipelineOutput, image,  has_nsfw_concept, None