class_docstr: Tortoise model class.
  
  Currently only supports inference.
  
  Examples:
      >>> from TTS.tts.configs.tortoise_config import TortoiseConfig
      >>> from TTS.tts.models.tortoise import Tortoise
      >>> config = TortoiseConfig()
      >>> model = Tortoise.inif_from_config(config)
      >>> model.load_checkpoint(config, checkpoint_dir="paths/to/models_dir/", eval=True)
class_docstr_tok: ["Tortoise","model","class",".","Currently","only","supports","inference",".","Examples",":",">",">",">","from","TTS.tts.configs.tortoise_config","import","TortoiseConfig",">",">",">","from","TTS.tts.models.tortoise","import","Tortoise",">",">",">","config","=","TortoiseConfig","(",")",">",">",">","model","=","Tortoise.inif_from_config","(","config",")",">",">",">","model.load_checkpoint","(","config",",","checkpoint_dir=","''","paths\/to\/models_dir\/","''",",","eval=True",")"]
docstr: This function produces an audio clip of the given text being spoken with the given reference voice.
  
  Args:
      text: (str) Text to be spoken.
      voice_samples: (List[Tuple[torch.Tensor]]) List of an arbitrary number of reference clips, which should be tuple-pairs
          of torch tensors containing arbitrary kHz waveform data.
      conditioning_latents: (Tuple[autoregressive_conditioning_latent, diffusion_conditioning_latent]) A tuple of
          (autoregressive_conditioning_latent, diffusion_conditioning_latent), which can be provided in lieu
          of voice_samples. This is ignored unless `voice_samples=None`. Conditioning latents can be retrieved
          via `get_conditioning_latents()`.
      k: (int) The number of returned clips. The most likely (as determined by Tortoises' CLVP model) clips are returned.
          latent_averaging_mode: (int) 0/1/2 for following modes:
          0 - latents will be generated as in original tortoise, using ~4.27s from each voice sample, averaging latent across all samples
          1 - latents will be generated using (almost) entire voice samples, averaged across all the ~4.27s chunks
          2 - latents will be generated using (almost) entire voice samples, averaged per voice sample
      verbose: (bool) Whether or not to print log messages indicating the progress of creating a clip. Default=true.
      num_autoregressive_samples: (int) Number of samples taken from the autoregressive model, all of which are filtered using CLVP.
          As Tortoise is a probabilistic model, more samples means a higher probability of creating something "great".
      temperature: (float) The softmax temperature of the autoregressive model.
      length_penalty: (float) A length penalty applied to the autoregressive decoder. Higher settings causes the model to produce more terse outputs.
      repetition_penalty: (float) A penalty that prevents the autoregressive decoder from repeating itself during decoding. Can be used to reduce
          the incidence of long silences or "uhhhhhhs", etc.
      top_p: (float) P value used in nucleus sampling. (0,1]. Lower values mean the decoder produces more "likely" (aka boring) outputs.
      max_mel_tokens: (int) Restricts the output length. (0,600] integer. Each unit is 1/20 of a second.
      typical_sampling: (bool) Turns typical sampling on or off. This sampling mode is discussed in this paper: https://arxiv.org/abs/2202.00666
          I was interested in the premise, but the results were not as good as I was hoping. This is off by default, but could use some tuning.
      typical_mass: (float) The typical_mass parameter from the typical_sampling algorithm.
      diffusion_iterations: (int) Number of diffusion steps to perform. [0,4000]. More steps means the network has more chances to iteratively
          refine the output, which should theoretically mean a higher quality output. Generally a value above 250 is not noticeably better, however.
      cond_free: (bool) Whether or not to perform conditioning-free diffusion. Conditioning-free diffusion performs two forward passes for
          each diffusion step: one with the outputs of the autoregressive model and one with no conditioning priors. The output of the two
          is blended according to the cond_free_k value below. Conditioning-free diffusion is the real deal, and dramatically improves realism.
      cond_free_k: (float) Knob that determines how to balance the conditioning free signal with the conditioning-present signal. [0,inf].
          As cond_free_k increases, the output becomes dominated by the conditioning-free signal.
      diffusion_temperature: (float) Controls the variance of the noise fed into the diffusion model. [0,1]. Values at 0
                                are the "mean" prediction of the diffusion network and will sound bland and smeared.
      hf_generate_kwargs: (**kwargs) The huggingface Transformers generate API is used for the autoregressive transformer.
                              Extra keyword args fed to this function get forwarded directly to that API. Documentation
                              here: https://huggingface.co/docs/transformers/internal/generation_utils
  
  Returns:
      Generated audio clip(s) as a torch tensor. Shape 1,S if k=1 else, (k,1,S) where S is the sample length.
      Sample rate is 24kHz.
docstr_tok: ["This","function","produces","an","audio","clip","of","the","given","text","being","spoken","with","the","given","reference","voice",".","Args",":","text",":","(","str",")","Text","to","be","spoken",".","voice_samples",":","(","List","[","Tuple","[","torch.Tensor","]","]",")","List","of","an","arbitrary","number","of","reference","clips",",","which","should","be","tuple-pairs","of","torch","tensors","containing","arbitrary","kHz","waveform","data",".","conditioning_latents",":","(","Tuple","[","autoregressive_conditioning_latent",",","diffusion_conditioning_latent","]",")","A","tuple","of","(","autoregressive_conditioning_latent",",","diffusion_conditioning_latent",")",",","which","can","be","provided","in","lieu","of","voice_samples",".","This","is","ignored","unless","`","voice_samples=None","`",".","Conditioning","latents","can","be","retrieved","via","`","get_conditioning_latents","(",")","`",".","k",":","(","int",")","The","number","of","returned","clips",".","The","most","likely","(","as","determined","by","Tortoises","'","CLVP","model",")","clips","are","returned",".","latent_averaging_mode",":","(","int",")","0\/1\/2","for","following","modes",":","0","-","latents","will","be","generated","as","in","original","tortoise",",","using","~4.27s","from","each","voice","sample",",","averaging","latent","across","all","samples","1","-","latents","will","be","generated","using","(","almost",")","entire","voice","samples",",","averaged","across","all","the","~4.27s","chunks","2","-","latents","will","be","generated","using","(","almost",")","entire","voice","samples",",","averaged","per","voice","sample","verbose",":","(","bool",")","Whether","or","not","to","print","log","messages","indicating","the","progress","of","creating","a","clip",".","Default=true",".","num_autoregressive_samples",":","(","int",")","Number","of","samples","taken","from","the","autoregressive","model",",","all","of","which","are","filtered","using","CLVP",".","As","Tortoise","is","a","probabilistic","model",",","more","samples","means","a","higher","probability","of","creating","something","``","great","''",".","temperature",":","(","float",")","The","softmax","temperature","of","the","autoregressive","model",".","length_penalty",":","(","float",")","A","length","penalty","applied","to","the","autoregressive","decoder",".","Higher","settings","causes","the","model","to","produce","more","terse","outputs",".","repetition_penalty",":","(","float",")","A","penalty","that","prevents","the","autoregressive","decoder","from","repeating","itself","during","decoding",".","Can","be","used","to","reduce","the","incidence","of","long","silences","or","``","uhhhhhhs","''",",","etc",".","top_p",":","(","float",")","P","value","used","in","nucleus","sampling",".","(","0,1","]",".","Lower","values","mean","the","decoder","produces","more","``","likely","''","(","aka","boring",")","outputs",".","max_mel_tokens",":","(","int",")","Restricts","the","output","length",".","(","0,600","]","integer",".","Each","unit","is","1\/20","of","a","second",".","typical_sampling",":","(","bool",")","Turns","typical","sampling","on","or","off",".","This","sampling","mode","is","discussed","in","this","paper",":","https",":","\/\/arxiv.org\/abs\/2202.00666","I","was","interested","in","the","premise",",","but","the","results","were","not","as","good","as","I","was","hoping",".","This","is","off","by","default",",","but","could","use","some","tuning",".","typical_mass",":","(","float",")","The","typical_mass","parameter","from","the","typical_sampling","algorithm",".","diffusion_iterations",":","(","int",")","Number","of","diffusion","steps","to","perform",".","[","0,4000","]",".","More","steps","means","the","network","has","more","chances","to","iteratively","refine","the","output",",","which","should","theoretically","mean","a","higher","quality","output",".","Generally","a","value","above","250","is","not","noticeably","better",",","however",".","cond_free",":","(","bool",")","Whether","or","not","to","perform","conditioning-free","diffusion",".","Conditioning-free","diffusion","performs","two","forward","passes","for","each","diffusion","step",":","one","with","the","outputs","of","the","autoregressive","model","and","one","with","no","conditioning","priors",".","The","output","of","the","two","is","blended","according","to","the","cond_free_k","value","below",".","Conditioning-free","diffusion","is","the","real","deal",",","and","dramatically","improves","realism",".","cond_free_k",":","(","float",")","Knob","that","determines","how","to","balance","the","conditioning","free","signal","with","the","conditioning-present","signal",".","[","0",",","inf","]",".","As","cond_free_k","increases",",","the","output","becomes","dominated","by","the","conditioning-free","signal",".","diffusion_temperature",":","(","float",")","Controls","the","variance","of","the","noise","fed","into","the","diffusion","model",".","[","0,1","]",".","Values","at","0","are","the","``","mean","''","prediction","of","the","diffusion","network","and","will","sound","bland","and","smeared",".","hf_generate_kwargs",":","(","*","*","kwargs",")","The","huggingface","Transformers","generate","API","is","used","for","the","autoregressive","transformer",".","Extra","keyword","args","fed","to","this","function","get","forwarded","directly","to","that","API",".","Documentation","here",":","https",":","\/\/huggingface.co\/docs\/transformers\/internal\/generation_utils","Returns",":","Generated","audio","clip","(","s",")","as","a","torch","tensor",".","Shape","1",",","S","if","k=1","else",",","(","k,1",",","S",")","where","S","is","the","sample","length",".","Sample","rate","is","24kHz","."]
code: def inference(
      self,
      text,
      voice_samples=None,
      conditioning_latents=None,
      k=1,
      verbose=True,
      use_deterministic_seed=None,
      return_deterministic_state=False,
      latent_averaging_mode=0,
      # autoregressive generation parameters follow
      num_autoregressive_samples=16,
      temperature=0.8,
      length_penalty=1,
      repetition_penalty=2.0,
      top_p=0.8,
      max_mel_tokens=500,
      # diffusion generation parameters follow
      diffusion_iterations=100,
      cond_free=True,
      cond_free_k=2,
      diffusion_temperature=1.0,
      sampler="ddim",
      half=True,
      original_tortoise=False,
      **hf_generate_kwargs,
  ):
      """
      This function produces an audio clip of the given text being spoken with the given reference voice.
  
      Args:
          text: (str) Text to be spoken.
          voice_samples: (List[Tuple[torch.Tensor]]) List of an arbitrary number of reference clips, which should be tuple-pairs
              of torch tensors containing arbitrary kHz waveform data.
          conditioning_latents: (Tuple[autoregressive_conditioning_latent, diffusion_conditioning_latent]) A tuple of
              (autoregressive_conditioning_latent, diffusion_conditioning_latent), which can be provided in lieu
              of voice_samples. This is ignored unless `voice_samples=None`. Conditioning latents can be retrieved
              via `get_conditioning_latents()`.
          k: (int) The number of returned clips. The most likely (as determined by Tortoises' CLVP model) clips are returned.
              latent_averaging_mode: (int) 0/1/2 for following modes:
              0 - latents will be generated as in original tortoise, using ~4.27s from each voice sample, averaging latent across all samples
              1 - latents will be generated using (almost) entire voice samples, averaged across all the ~4.27s chunks
              2 - latents will be generated using (almost) entire voice samples, averaged per voice sample
          verbose: (bool) Whether or not to print log messages indicating the progress of creating a clip. Default=true.
          num_autoregressive_samples: (int) Number of samples taken from the autoregressive model, all of which are filtered using CLVP.
              As Tortoise is a probabilistic model, more samples means a higher probability of creating something "great".
          temperature: (float) The softmax temperature of the autoregressive model.
          length_penalty: (float) A length penalty applied to the autoregressive decoder. Higher settings causes the model to produce more terse outputs.
          repetition_penalty: (float) A penalty that prevents the autoregressive decoder from repeating itself during decoding. Can be used to reduce
              the incidence of long silences or "uhhhhhhs", etc.
          top_p: (float) P value used in nucleus sampling. (0,1]. Lower values mean the decoder produces more "likely" (aka boring) outputs.
          max_mel_tokens: (int) Restricts the output length. (0,600] integer. Each unit is 1/20 of a second.
          typical_sampling: (bool) Turns typical sampling on or off. This sampling mode is discussed in this paper: https://arxiv.org/abs/2202.00666
              I was interested in the premise, but the results were not as good as I was hoping. This is off by default, but could use some tuning.
          typical_mass: (float) The typical_mass parameter from the typical_sampling algorithm.
          diffusion_iterations: (int) Number of diffusion steps to perform. [0,4000]. More steps means the network has more chances to iteratively
              refine the output, which should theoretically mean a higher quality output. Generally a value above 250 is not noticeably better, however.
          cond_free: (bool) Whether or not to perform conditioning-free diffusion. Conditioning-free diffusion performs two forward passes for
              each diffusion step: one with the outputs of the autoregressive model and one with no conditioning priors. The output of the two
              is blended according to the cond_free_k value below. Conditioning-free diffusion is the real deal, and dramatically improves realism.
          cond_free_k: (float) Knob that determines how to balance the conditioning free signal with the conditioning-present signal. [0,inf].
              As cond_free_k increases, the output becomes dominated by the conditioning-free signal.
          diffusion_temperature: (float) Controls the variance of the noise fed into the diffusion model. [0,1]. Values at 0
                                    are the "mean" prediction of the diffusion network and will sound bland and smeared.
          hf_generate_kwargs: (**kwargs) The huggingface Transformers generate API is used for the autoregressive transformer.
                                  Extra keyword args fed to this function get forwarded directly to that API. Documentation
                                  here: https://huggingface.co/docs/transformers/internal/generation_utils
  
      Returns:
          Generated audio clip(s) as a torch tensor. Shape 1,S if k=1 else, (k,1,S) where S is the sample length.
          Sample rate is 24kHz.
      """
      deterministic_seed = deterministic_state(
          seed=use_deterministic_seed
      )
  
      text_tokens = (
          torch.IntTensor(self.tokenizer.encode(text))
          .unsqueeze(0)
          .to(self.device)
      )
      text_tokens = F.pad(
          text_tokens, (0, 1)
      )  # This may not be necessary.
      assert (
          text_tokens.shape[-1] < 400
      ), "Too much text provided. Break the text up into separate segments and re-try inference."
  
      if voice_samples is not None:
          (
              auto_conditioning,
              diffusion_conditioning,
              _,
              _,
          ) = self.get_conditioning_latents(
              voice_samples,
              return_mels=True,
              latent_averaging_mode=latent_averaging_mode,
              original_tortoise=original_tortoise,
          )
      elif conditioning_latents is not None:
          (
              auto_conditioning,
              diffusion_conditioning,
          ) = conditioning_latents
      else:
          (
              auto_conditioning,
              diffusion_conditioning,
          ) = self.get_random_conditioning_latents()
      auto_conditioning = auto_conditioning.to(self.device)
      diffusion_conditioning = diffusion_conditioning.to(self.device)
  
      diffuser = load_discrete_vocoder_diffuser(
          desired_diffusion_steps=diffusion_iterations,
          cond_free=cond_free,
          cond_free_k=cond_free_k,
          sampler=sampler,
      )
  
      # in the case of single_sample,
      orig_batch_size = self.autoregressive_batch_size
      while num_autoregressive_samples % self.autoregressive_batch_size:
          self.autoregressive_batch_size //= 2
      with torch.no_grad():
          samples = []
          num_batches = (
              num_autoregressive_samples
              // self.autoregressive_batch_size
          )
          stop_mel_token = self.autoregressive.stop_mel_token
          calm_token = 83  # This is the token for coding silence, which is fixed in place with "fix_autoregressive_output"
          self.autoregressive = self.autoregressive.to(self.device)
          if verbose:
              print("Generating autoregressive samples..")
          with self.temporary_cuda(
              self.autoregressive
          ) as autoregressive, torch.autocast(
              device_type="cuda", dtype=torch.float16, enabled=half
          ):
              for b in tqdm(range(num_batches), disable=not verbose):
                  codes = autoregressive.inference_speech(
                      auto_conditioning,
                      text_tokens,
                      do_sample=True,
                      top_p=top_p,
                      temperature=temperature,
                      num_return_sequences=self.autoregressive_batch_size,
                      length_penalty=length_penalty,
                      repetition_penalty=repetition_penalty,
                      max_generate_length=max_mel_tokens,
                      **hf_generate_kwargs,
                  )
                  padding_needed = max_mel_tokens - codes.shape[1]
                  codes = F.pad(
                      codes, (0, padding_needed), value=stop_mel_token
                  )
                  samples.append(codes)
          self.autoregressive_batch_size = (
              orig_batch_size  # in the case of single_sample
          )
  
          clip_results = []
          with self.temporary_cuda(self.clvp) as clvp, torch.autocast(
              device_type="cuda", dtype=torch.float16, enabled=half
          ):
              for batch in tqdm(samples, disable=not verbose):
                  for i in range(batch.shape[0]):
                      batch[i] = fix_autoregressive_output(
                          batch[i], stop_mel_token
                      )
                  clvp_res = clvp(
                      text_tokens.repeat(batch.shape[0], 1),
                      batch,
                      return_loss=False,
                  )
                  clip_results.append(clvp_res)
  
              clip_results = torch.cat(clip_results, dim=0)
              samples = torch.cat(samples, dim=0)
              best_results = samples[
                  torch.topk(clip_results, k=k).indices
              ]
          del samples
  
          # The diffusion model actually wants the last hidden layer from the autoregressive model as conditioning
          # inputs. Re-produce those for the top results. This could be made more efficient by storing all of these
          # results, but will increase memory usage.
          with self.temporary_cuda(
              self.autoregressive
          ) as autoregressive:
              best_latents = autoregressive(
                  auto_conditioning.repeat(k, 1),
                  text_tokens.repeat(k, 1),
                  torch.tensor(
                      [text_tokens.shape[-1]], device=text_tokens.device
                  ),
                  best_results,
                  torch.tensor(
                      [
                          best_results.shape[-1]
                          * self.autoregressive.mel_length_compression
                      ],
                      device=text_tokens.device,
                  ),
                  return_latent=True,
                  clip_inputs=False,
              )
          del auto_conditioning
  
          if verbose:
              print("Transforming autoregressive outputs into audio..")
          wav_candidates = []
          for b in range(best_results.shape[0]):
              codes = best_results[b].unsqueeze(0)
              latents = best_latents[b].unsqueeze(0)
  
              # Find the first occurrence of the "calm" token and trim the codes to that.
              ctokens = 0
              for code in range(codes.shape[-1]):
                  if codes[0, code] == calm_token:
                      ctokens += 1
                  else:
                      ctokens = 0
                  if (
                      ctokens > 8
                  ):  # 8 tokens gives the diffusion model some "breathing room" to terminate speech.
                      latents = latents[:, :code]
                      break
              with self.temporary_cuda(self.diffusion) as diffusion:
                  mel = do_spectrogram_diffusion(
                      diffusion,
                      diffuser,
                      latents,
                      diffusion_conditioning,
                      temperature=diffusion_temperature,
                      verbose=verbose,
                  )
              with self.temporary_cuda(self.vocoder) as vocoder:
                  wav = vocoder.inference(mel)
                  wav_candidates.append(wav.cpu())
  
          def potentially_redact(clip, text):
              if self.enable_redaction:
                  return self.aligner.redact(
                      clip.squeeze(1), text
                  ).unsqueeze(1)
              return clip
  
          wav_candidates = [
              potentially_redact(wav_candidate, text)
              for wav_candidate in wav_candidates
          ]
  
          if len(wav_candidates) > 1:
              res = wav_candidates
          else:
              res = wav_candidates[0]
  
      return_dict = {
          "wav": res,
          "deterministic_seed": None,
          "text": None,
          "voice_samples": None,
          "conditioning_latents": None,
      }
      if return_deterministic_state:
          return_dict = {
              "wav": res,
              "deterministic_seed": deterministic_seed,
              "text": text,
              "voice_samples": voice_samples,
              "conditioning_latents": conditioning_latents,
          }
      return return_dict
  
imports: import os
  import random
  from contextlib import contextmanager
  from dataclasses import dataclass
  from time import time
  import torch
  import torch.nn.functional
  import torchaudio
  from coqpit import Coqpit
  from tqdm import tqdm
  from TTS.tts.layers.tortoise.arch_utils import TorchMelSpectrogram
  from TTS.tts.layers.tortoise.audio_utils import denormalize_tacotron_mel, load_voice, wav_to_univnet_mel
  from TTS.tts.layers.tortoise.autoregressive import UnifiedVoice
  from TTS.tts.layers.tortoise.classifier import AudioMiniEncoderWithClassifierHead
  from TTS.tts.layers.tortoise.clvp import CLVP
  from TTS.tts.layers.tortoise.diffusion import SpacedDiffusion, get_named_beta_schedule, space_timesteps
  from TTS.tts.layers.tortoise.diffusion_decoder import DiffusionTts
  from TTS.tts.layers.tortoise.random_latent_generator import RandomLatentConverter
  from TTS.tts.layers.tortoise.tokenizer import VoiceBpeTokenizer
  from TTS.tts.layers.tortoise.vocoder import VocConf, VocType
  from TTS.tts.layers.tortoise.wav2vec_alignment import Wav2VecAlignment
  from TTS.tts.models.base_tts import BaseTTS
label_desc: Use image node_id 7 for calling the Tortoise obj's underlying member method code with example usage: obj.inference(text, voice_samples, conditioning_latents, k, verbose, use_deterministic_seed, return_deterministic_state, latent_averaging_mode, num_autoregressive_samples, temperature, length_penalty, repetition_penalty, top_p, max_mel_tokens, diffusion_iterations, cond_free, cond_free_k, diffusion_temperature, sampler, half, original_tortoise) and returns: return_dict, clip, self