class_docstr: High-level API for creating images of pileups of reads and reference bases.
  
  This class provides a higher-level and more natural API for constructing
  images at a candidate variant call site. Given a DeepVariantCall, which
  contains the candidate variant call along with key supplementary information,
  this class provides create_pileup_images() that will do all of the necessary
  fetching of reads and reference bases from readers and pass those off to the
  lower-level PileupImageEncoder to construct the image Tensor.
  
  for dv_call in candidates:
    allele_and_images = pic.create_pileup_images(dv_call)
    ...
  
  A quick note on how we deal with multiple alt alleles:
  
  Suppose variant has ref and two alt alleles. Assuming the sample is diploid,
  we have the following six possible genotypes:
  
    ref/ref   => 0/0
    ref/alt1  => 0/1
    alt1/alt1 => 1/1
    ref/alt2  => 0/2
    alt1/alt2 => 1/2
    alt2/alt2 => 2/2
  
  In DeepVariant we predict the genotype count (0, 1, 2) for a specific set of
  alternate alleles. If we only had a single alt, we'd construct an image for
  ref vs. alt1:
  
    image1 => ref vs. alt1 => determine if we are 0/0, 0/1, 1/1
  
  If we add a second image for alt2, we get:
  
    image2 => ref vs. alt2 => determine if we are 0/0, 0/2, 2/2
  
  but the problem here is that we don't have a good estimate for the het-alt
  state 1/2. So we construct a third image contrasting ref vs. either alt1 or
  alt2:
  
    image3 => ref vs. alt1 or alt2 => determines 0/0, 0/{1,2}, {1,2}/{1,2}
  
  Given the predictions for each image:
  
    image1 => p00, p01, p11
    image2 => p00, p02, p22
    image3 => p00, p0x, pxx where x is {1,2}
  
  we calculate our six genotype likelihoods as:
  
    0/0 => p00 [from any image]
    0/1 => p01 [image1]
    1/1 => p11 [image1]
    0/2 => p02 [image2]
    2/2 => p22 [image2]
    1/2 => pxx [image3]
  
  The function create_pileup_images() returns all of the necessary images, along
  with the alt alleles used for each image.
class_docstr_tok: ["High-level","API","for","creating","images","of","pileups","of","reads","and","reference","bases",".","This","class","provides","a","higher-level","and","more","natural","API","for","constructing","images","at","a","candidate","variant","call","site",".","Given","a","DeepVariantCall",",","which","contains","the","candidate","variant","call","along","with","key","supplementary","information",",","this","class","provides","create_pileup_images","(",")","that","will","do","all","of","the","necessary","fetching","of","reads","and","reference","bases","from","readers","and","pass","those","off","to","the","lower-level","PileupImageEncoder","to","construct","the","image","Tensor",".","for","dv_call","in","candidates",":","allele_and_images","=","pic.create_pileup_images","(","dv_call",")","...","A","quick","note","on","how","we","deal","with","multiple","alt","alleles",":","Suppose","variant","has","ref","and","two","alt","alleles",".","Assuming","the","sample","is","diploid",",","we","have","the","following","six","possible","genotypes",":","ref\/ref","=",">","0\/0","ref\/alt1","=",">","0\/1","alt1\/alt1","=",">","1\/1","ref\/alt2","=",">","0\/2","alt1\/alt2","=",">","1\/2","alt2\/alt2","=",">","2\/2","In","DeepVariant","we","predict","the","genotype","count","(","0",",","1",",","2",")","for","a","specific","set","of","alternate","alleles",".","If","we","only","had","a","single","alt",",","we","'d","construct","an","image","for","ref","vs.","alt1",":","image1","=",">","ref","vs.","alt1","=",">","determine","if","we","are","0\/0",",","0\/1",",","1\/1","If","we","add","a","second","image","for","alt2",",","we","get",":","image2","=",">","ref","vs.","alt2","=",">","determine","if","we","are","0\/0",",","0\/2",",","2\/2","but","the","problem","here","is","that","we","do","n't","have","a","good","estimate","for","the","het-alt","state","1\/2",".","So","we","construct","a","third","image","contrasting","ref","vs.","either","alt1","or","alt2",":","image3","=",">","ref","vs.","alt1","or","alt2","=",">","determines","0\/0",",","0\/","{","1,2","}",",","{","1,2","}","\/","{","1,2","}","Given","the","predictions","for","each","image",":","image1","=",">","p00",",","p01",",","p11","image2","=",">","p00",",","p02",",","p22","image3","=",">","p00",",","p0x",",","pxx","where","x","is","{","1,2","}","we","calculate","our","six","genotype","likelihoods","as",":","0\/0","=",">","p00","[","from","any","image","]","0\/1","=",">","p01","[","image1","]","1\/1","=",">","p11","[","image1","]","0\/2","=",">","p02","[","image2","]","2\/2","=",">","p22","[","image2","]","1\/2","=",">","pxx","[","image3","]","The","function","create_pileup_images","(",")","returns","all","of","the","necessary","images",",","along","with","the","alt","alleles","used","for","each","image","."]
docstr: Creates a pileup tensor for dv_call.
  
  Args:
    dv_call: learning.genomics.deepvariant.DeepVariantCall object with
      information on our candidate call and allele support information.
    refbases: A string options.width in length containing the reference base
      sequence to encode. The middle base of this string should be at the
      start of the variant in dv_call.
    reads_for_samples: list by sample of Iterable of
      third_party.nucleus.protos.Read objects that we'll use to encode the
      read information supporting our call. Assumes each read is aligned and
      is well-formed (e.g., has bases and quality scores, cigar). Rows of the
      image are encoded in the same order as reads.
    alt_alleles: A collection of alternative_bases from dv_call.variant that
      we are treating as "alt" when constructing this pileup image. A read
      will be considered supporting the "alt" allele if it occurs in the
      support list for any alt_allele in this collection.
    sample_order: A list of indices representing the order in which samples
      should be represented in the pileup image. Example: [1,0,2] to swap the
      first two samples out of three. This is None by default which puts the
      samples in order.
    custom_ref: True if refbases should not be checked for matching against
      variant's reference_bases.
  
  Returns:
    A uint8 Tensor image of shape
      [self.width, <sum of sample pileup heights>, DEFAULT_NUM_CHANNEL]
  
  Raises:
    ValueError: if any arguments are invalid.
docstr_tok: ["Creates","a","pileup","tensor","for","dv_call",".","Args",":","dv_call",":","learning.genomics.deepvariant.DeepVariantCall","object","with","information","on","our","candidate","call","and","allele","support","information",".","refbases",":","A","string","options.width","in","length","containing","the","reference","base","sequence","to","encode",".","The","middle","base","of","this","string","should","be","at","the","start","of","the","variant","in","dv_call",".","reads_for_samples",":","list","by","sample","of","Iterable","of","third_party.nucleus.protos.Read","objects","that","we","'ll","use","to","encode","the","read","information","supporting","our","call",".","Assumes","each","read","is","aligned","and","is","well-formed","(","e.g.",",","has","bases","and","quality","scores",",","cigar",")",".","Rows","of","the","image","are","encoded","in","the","same","order","as","reads",".","alt_alleles",":","A","collection","of","alternative_bases","from","dv_call.variant","that","we","are","treating","as","``","alt","''","when","constructing","this","pileup","image",".","A","read","will","be","considered","supporting","the","``","alt","''","allele","if","it","occurs","in","the","support","list","for","any","alt_allele","in","this","collection",".","sample_order",":","A","list","of","indices","representing","the","order","in","which","samples","should","be","represented","in","the","pileup","image",".","Example",":","[","1,0,2","]","to","swap","the","first","two","samples","out","of","three",".","This","is","None","by","default","which","puts","the","samples","in","order",".","custom_ref",":","True","if","refbases","should","not","be","checked","for","matching","against","variant","'s","reference_bases",".","Returns",":","A","uint8","Tensor","image","of","shape","[","self.width",",","<","sum","of","sample","pileup","heights",">",",","DEFAULT_NUM_CHANNEL","]","Raises",":","ValueError",":","if","any","arguments","are","invalid","."]
code: def build_pileup(
      self,
      dv_call: deepvariant_pb2.DeepVariantCall,
      refbases: str,
      reads_for_samples: List[List[reads_pb2.Read]],
      alt_alleles: List[str],
      sample_order: Optional[List[int]] = None,
      custom_ref: bool = False,
  ):
      """Creates a pileup tensor for dv_call.
  
      Args:
        dv_call: learning.genomics.deepvariant.DeepVariantCall object with
          information on our candidate call and allele support information.
        refbases: A string options.width in length containing the reference base
          sequence to encode. The middle base of this string should be at the
          start of the variant in dv_call.
        reads_for_samples: list by sample of Iterable of
          third_party.nucleus.protos.Read objects that we'll use to encode the
          read information supporting our call. Assumes each read is aligned and
          is well-formed (e.g., has bases and quality scores, cigar). Rows of the
          image are encoded in the same order as reads.
        alt_alleles: A collection of alternative_bases from dv_call.variant that
          we are treating as "alt" when constructing this pileup image. A read
          will be considered supporting the "alt" allele if it occurs in the
          support list for any alt_allele in this collection.
        sample_order: A list of indices representing the order in which samples
          should be represented in the pileup image. Example: [1,0,2] to swap the
          first two samples out of three. This is None by default which puts the
          samples in order.
        custom_ref: True if refbases should not be checked for matching against
          variant's reference_bases.
  
      Returns:
        A uint8 Tensor image of shape
          [self.width, <sum of sample pileup heights>, DEFAULT_NUM_CHANNEL]
  
      Raises:
        ValueError: if any arguments are invalid.
      """
      if len(refbases) != self.width:
          raise ValueError(
              "refbases is {} long but width is {}".format(
                  len(refbases), self.width
              )
          )
  
      if not alt_alleles:
          raise ValueError("alt_alleles cannot be empty")
      if any(
          alt not in dv_call.variant.alternate_bases
          for alt in alt_alleles
      ):
          raise ValueError(
              (
                  "all elements of alt_alleles must be the alternate bases"
                  " of dv_call.variant"
              ),
              alt_alleles,
              dv_call.variant,
          )
      if len(self._samples) != len(reads_for_samples):
          raise ValueError(
              "The number of self._samples ({}) must be the same as the number of "
              "reads_for_samples ({}).".format(
                  len(self._samples), len(reads_for_samples)
              )
          )
  
      image_start_pos = dv_call.variant.start - self.half_width
      if not custom_ref and (
          refbases[self.half_width]
          != dv_call.variant.reference_bases[0]
      ):
          raise ValueError(
              "The middle base of reference sequence in the window "
              "({} at base {}) doesn't match first "
              "character of variant.reference_bases ({}).".format(
                  refbases[self.half_width],
                  self.half_width,
                  dv_call.variant.reference_bases,
              )
          )
  
      def build_pileup_for_one_sample(
          reads: List[reads_pb2.Read], sample: sample_lib.Sample
      ) -> List[np.ndarray]:
          """Create read pileup image section for one sample."""
          # We start with n copies of our encoded reference bases.
          rows = [
              self._encoder.encode_reference(refbases)
          ] * self.reference_band_height
  
          def _update_hap_index(
              read: reads_pb2.Read, hp_tag_for_assembly_polishing: int
          ) -> int:
              default_hap_idx = (
                  0  # By default, reads with no HP is set to 0.
              )
              if "HP" not in read.info:
                  return default_hap_idx
              read_info_hp = read.info.get("HP")
              if read_info_hp is None or not read_info_hp.values:
                  return default_hap_idx
              hp_field = next(iter(read_info_hp.values))
              if not hp_field.HasField("int_value"):
                  return default_hap_idx
              hp_value = hp_field.int_value
              if (
                  hp_tag_for_assembly_polishing > 0
                  and hp_value == hp_tag_for_assembly_polishing
              ):
                  # For the target HP tag, set it to -1 so it will be sorted on
                  # top of the pileup image.
                  return -1
              elif hp_value < 0:
                  return 0  # For reads with HP < 0, assume it is not tagged.
              else:
                  return hp_value
  
          # Returns tuples of the form (haplotype, position, row),
          # if the read can be encoded as a valid row to be used in the pileup
          # image.
          def _row_helper(
              read: reads_pb2.Read,
          ) -> Optional[Tuple[int, int, np.ndarray]]:
              """A function that returns tuples of (haplotype, position, row)."""
              read_row = self._encoder.encode_read(
                  dv_call, refbases, read, image_start_pos, alt_alleles
              )
              if read_row is None:
                  return None
              hap_idx = 0  # By default, reads with no HP is set to 0.
              if self._options.sort_by_haplotypes:
                  hap_idx = _update_hap_index(
                      read, self._options.hp_tag_for_assembly_polishing
                  )
              return hap_idx, read.alignment.position.position, read_row
  
          # We add a row for each read in order, down-sampling if the number of
          # reads is greater than the max reads for each sample. Sort the reads by
          # their alignment position.
          random_for_image = np.random.RandomState(
              self._options.random_seed
          )
  
          # Use sample height or default to pic height.
          if sample.options.pileup_height != 0:
              pileup_height = sample.options.pileup_height
          else:
              pileup_height = self.height
  
          max_reads = pileup_height - self.reference_band_height
          reads_indices = list(range(len(reads)))
          if len(reads) > max_reads:
              # Shuffle the indices instead of the reads, so that we won't change
              # the order of the reads list.
              random_for_image.shuffle(reads_indices)
          pileup_of_reads = []
          for reads_index in reads_indices:
              read = reads[reads_index]
              if len(pileup_of_reads) == max_reads:
                  break
              row = _row_helper(read)
              if row is None:
                  continue
              pileup_of_reads.append(row)
          pileup_of_reads = sorted(
              pileup_of_reads, key=lambda x: (x[0], x[1])
          )
  
          rows += [read_row for _, _, read_row in pileup_of_reads]
  
          # Finally, fill in any missing rows to bring our image to pileup_height
          # rows with empty (all black) pixels.
          n_missing_rows = pileup_height - len(rows)
          if n_missing_rows > 0:
              # Add values to rows to fill it out with zeros.
              rows += [self._empty_image_row()] * n_missing_rows
          return rows
  
      sample_sections = []
      if sample_order is None:
          sample_order = range(len(self._samples))
      for i in sample_order:
          sample = self._samples[i]
          sample_sections.extend(
              build_pileup_for_one_sample(reads_for_samples[i], sample)
          )
  
      # Vertically stack the image rows to create a single
      # h x w x DEFAULT_NUM_CHANNEL image.
      return np.vstack(sample_sections)
  
imports: import itertools
  from typing import Iterable, List, Optional, Tuple
  from absl import logging
  import numpy
  from deepvariant import dv_constants
  from deepvariant import sample
  from deepvariant.protos import deepvariant_pb2
  from deepvariant.python import pileup_image_native
  from third_party.nucleus.io import fasta
  from third_party.nucleus.io import sam
  from third_party.nucleus.protos import reads_pb2
  from third_party.nucleus.protos import variants_pb2
  from third_party.nucleus.util import ranges
label_desc: Use image node_id 8 for calling the PileupImageCreator obj's underlying member method code with example usage: obj.build_pileup(dv_call, refbases, reads_for_samples, alt_alleles, sample_order, custom_ref) and returns: np, rows, hap_idx,  read,  read_row, default_hap_idx, default_hap_idx, default_hap_idx, unknown, None, int, hp_value