class_docstr: A class to build a FrameData object, load and process the binary data (crop and
  resize). This is an abstract class for extending to build FrameData subtypes. Most
  users need to use concrete `FrameDataBuilder` class instead.
  Beware that modifications of frame data are done in-place.
  
  Args:
      dataset_root: The root folder of the dataset; all paths in frame / sequence
          annotations are defined w.r.t. this root. Has to be set if any of the
          load_* flabs below is true.
      load_images: Enable loading the frame RGB data.
      load_depths: Enable loading the frame depth maps.
      load_depth_masks: Enable loading the frame depth map masks denoting the
          depth values used for evaluation (the points consistent across views).
      load_masks: Enable loading frame foreground masks.
      load_point_clouds: Enable loading sequence-level point clouds.
      max_points: Cap on the number of loaded points in the point cloud;
          if reached, they are randomly sampled without replacement.
      mask_images: Whether to mask the images with the loaded foreground masks;
          0 value is used for background.
      mask_depths: Whether to mask the depth maps with the loaded foreground
          masks; 0 value is used for background.
      image_height: The height of the returned images, masks, and depth maps;
          aspect ratio is preserved during cropping/resizing.
      image_width: The width of the returned images, masks, and depth maps;
          aspect ratio is preserved during cropping/resizing.
      box_crop: Enable cropping of the image around the bounding box inferred
          from the foreground region of the loaded segmentation mask; masks
          and depth maps are cropped accordingly; cameras are corrected.
      box_crop_mask_thr: The threshold used to separate pixels into foreground
          and background based on the foreground_probability mask; if no value
          is greater than this threshold, the loader lowers it and repeats.
      box_crop_context: The amount of additional padding added to each
          dimension of the cropping bounding box, relative to box size.
      path_manager: Optionally a PathManager for interpreting paths in a special way.
class_docstr_tok: ["A","class","to","build","a","FrameData","object",",","load","and","process","the","binary","data","(","crop","and","resize",")",".","This","is","an","abstract","class","for","extending","to","build","FrameData","subtypes",".","Most","users","need","to","use","concrete","`","FrameDataBuilder","`","class","instead",".","Beware","that","modifications","of","frame","data","are","done","in-place",".","Args",":","dataset_root",":","The","root","folder","of","the","dataset",";","all","paths","in","frame","\/","sequence","annotations","are","defined","w.r.t",".","this","root",".","Has","to","be","set","if","any","of","the","load_","*","flabs","below","is","true",".","load_images",":","Enable","loading","the","frame","RGB","data",".","load_depths",":","Enable","loading","the","frame","depth","maps",".","load_depth_masks",":","Enable","loading","the","frame","depth","map","masks","denoting","the","depth","values","used","for","evaluation","(","the","points","consistent","across","views",")",".","load_masks",":","Enable","loading","frame","foreground","masks",".","load_point_clouds",":","Enable","loading","sequence-level","point","clouds",".","max_points",":","Cap","on","the","number","of","loaded","points","in","the","point","cloud",";","if","reached",",","they","are","randomly","sampled","without","replacement",".","mask_images",":","Whether","to","mask","the","images","with","the","loaded","foreground","masks",";","0","value","is","used","for","background",".","mask_depths",":","Whether","to","mask","the","depth","maps","with","the","loaded","foreground","masks",";","0","value","is","used","for","background",".","image_height",":","The","height","of","the","returned","images",",","masks",",","and","depth","maps",";","aspect","ratio","is","preserved","during","cropping\/resizing",".","image_width",":","The","width","of","the","returned","images",",","masks",",","and","depth","maps",";","aspect","ratio","is","preserved","during","cropping\/resizing",".","box_crop",":","Enable","cropping","of","the","image","around","the","bounding","box","inferred","from","the","foreground","region","of","the","loaded","segmentation","mask",";","masks","and","depth","maps","are","cropped","accordingly",";","cameras","are","corrected",".","box_crop_mask_thr",":","The","threshold","used","to","separate","pixels","into","foreground","and","background","based","on","the","foreground_probability","mask",";","if","no","value","is","greater","than","this","threshold",",","the","loader","lowers","it","and","repeats",".","box_crop_context",":","The","amount","of","additional","padding","added","to","each","dimension","of","the","cropping","bounding","box",",","relative","to","box","size",".","path_manager",":","Optionally","a","PathManager","for","interpreting","paths","in","a","special","way","."]
docstr: Builds the frame data based on raw frame/sequence annotations, loads the
  binary data and adjust them according to the metadata. The processing includes:
      * if box_crop is set, the image/mask/depth are cropped with the bounding
          box provided or estimated from MaskAnnotation,
      * if image_height/image_width are set, the image/mask/depth are resized to
          fit that resolution. Note that the aspect ratio is preserved, and the
          (possibly cropped) image is pasted into the top-left corner. In the
          resulting frame_data, mask_crop field corresponds to the mask of the
          pasted image.
  
  Args:
      frame_annotation: frame annotation
      sequence_annotation: sequence annotation
      load_blobs: if the function should attempt loading the image, depth map
          and mask, and foreground mask
  
  Returns:
      The constructed FrameData object.
docstr_tok: ["Builds","the","frame","data","based","on","raw","frame\/sequence","annotations",",","loads","the","binary","data","and","adjust","them","according","to","the","metadata",".","The","processing","includes",":","*","if","box_crop","is","set",",","the","image\/mask\/depth","are","cropped","with","the","bounding","box","provided","or","estimated","from","MaskAnnotation",",","*","if","image_height\/image_width","are","set",",","the","image\/mask\/depth","are","resized","to","fit","that","resolution",".","Note","that","the","aspect","ratio","is","preserved",",","and","the","(","possibly","cropped",")","image","is","pasted","into","the","top-left","corner",".","In","the","resulting","frame_data",",","mask_crop","field","corresponds","to","the","mask","of","the","pasted","image",".","Args",":","frame_annotation",":","frame","annotation","sequence_annotation",":","sequence","annotation","load_blobs",":","if","the","function","should","attempt","loading","the","image",",","depth","map","and","mask",",","and","foreground","mask","Returns",":","The","constructed","FrameData","object","."]
code: def build(
      self,
      frame_annotation: types.FrameAnnotation,
      sequence_annotation: types.SequenceAnnotation,
      *,
      load_blobs: bool = True,
      **kwargs,
  ) -> FrameDataSubtype:
      """Builds the frame data based on raw frame/sequence annotations, loads the
      binary data and adjust them according to the metadata. The processing includes:
          * if box_crop is set, the image/mask/depth are cropped with the bounding
              box provided or estimated from MaskAnnotation,
          * if image_height/image_width are set, the image/mask/depth are resized to
              fit that resolution. Note that the aspect ratio is preserved, and the
              (possibly cropped) image is pasted into the top-left corner. In the
              resulting frame_data, mask_crop field corresponds to the mask of the
              pasted image.
  
      Args:
          frame_annotation: frame annotation
          sequence_annotation: sequence annotation
          load_blobs: if the function should attempt loading the image, depth map
              and mask, and foreground mask
  
      Returns:
          The constructed FrameData object.
      """
  
      point_cloud = sequence_annotation.point_cloud
  
      frame_data = self.frame_data_type(
          frame_number=safe_as_tensor(
              frame_annotation.frame_number, torch.long
          ),
          frame_timestamp=safe_as_tensor(
              frame_annotation.frame_timestamp, torch.float
          ),
          sequence_name=frame_annotation.sequence_name,
          sequence_category=sequence_annotation.category,
          camera_quality_score=safe_as_tensor(
              sequence_annotation.viewpoint_quality_score, torch.float
          ),
          point_cloud_quality_score=safe_as_tensor(
              point_cloud.quality_score, torch.float
          )
          if point_cloud is not None
          else None,
      )
  
      fg_mask_np: Optional[np.ndarray] = None
      mask_annotation = frame_annotation.mask
      if mask_annotation is not None:
          if load_blobs and self.load_masks:
              fg_mask_np, mask_path = self._load_fg_probability(
                  frame_annotation
              )
              frame_data.mask_path = mask_path
              frame_data.fg_probability = safe_as_tensor(
                  fg_mask_np, torch.float
              )
  
          bbox_xywh = mask_annotation.bounding_box_xywh
          if bbox_xywh is None and fg_mask_np is not None:
              bbox_xywh = get_bbox_from_mask(
                  fg_mask_np, self.box_crop_mask_thr
              )
  
          frame_data.bbox_xywh = safe_as_tensor(bbox_xywh, torch.float)
  
      if frame_annotation.image is not None:
          image_size_hw = safe_as_tensor(
              frame_annotation.image.size, torch.long
          )
          frame_data.image_size_hw = (
              image_size_hw  # original image size
          )
          # image size after crop/resize
          frame_data.effective_image_size_hw = image_size_hw
          image_path = None
          dataset_root = self.dataset_root
          if (
              frame_annotation.image.path is not None
              and dataset_root is not None
          ):
              image_path = os.path.join(
                  dataset_root, frame_annotation.image.path
              )
              frame_data.image_path = image_path
  
          if load_blobs and self.load_images:
              if image_path is None:
                  raise ValueError(
                      "Image path is required to load images."
                  )
  
              image_np = load_image(self._local_path(image_path))
              frame_data.image_rgb = self._postprocess_image(
                  image_np,
                  frame_annotation.image.size,
                  frame_data.fg_probability,
              )
  
      if (
          load_blobs
          and self.load_depths
          and frame_annotation.depth is not None
          and frame_annotation.depth.path is not None
      ):
          (
              frame_data.depth_map,
              frame_data.depth_path,
              frame_data.depth_mask,
          ) = self._load_mask_depth(frame_annotation, fg_mask_np)
  
      if (
          load_blobs
          and self.load_point_clouds
          and point_cloud is not None
      ):
          pcl_path = self._fix_point_cloud_path(point_cloud.path)
          frame_data.sequence_point_cloud = load_pointcloud(
              self._local_path(pcl_path), max_points=self.max_points
          )
          frame_data.sequence_point_cloud_path = pcl_path
  
      if frame_annotation.viewpoint is not None:
          frame_data.camera = self._get_pytorch3d_camera(
              frame_annotation
          )
  
      if self.box_crop:
          frame_data.crop_by_metadata_bbox_(self.box_crop_context)
  
      if self.image_height is not None and self.image_width is not None:
          new_size = (self.image_height, self.image_width)
          frame_data.resize_frame_(
              new_size_hw=torch.tensor(
                  new_size, dtype=torch.long
              ),  # pyre-ignore
          )
  
      return frame_data
  
imports: import os
  from abc import ABC, abstractmethod
  from collections import defaultdict
  from dataclasses import dataclass, field, fields
  from typing import Any, ClassVar, Generic, List, Mapping, Optional, Tuple, Type, TypeVar, Union
  import numpy
  import torch
  from pytorch3d.implicitron.dataset import types
  from pytorch3d.implicitron.dataset.utils import adjust_camera_to_bbox_crop_, adjust_camera_to_image_scale_, bbox_xyxy_to_xywh, clamp_box_to_image_bounds_and_round, crop_around_box, GenericWorkaround, get_bbox_from_mask, get_clamp_bbox, load_depth, load_depth_mask, load_image, load_mask, load_pointcloud, rescale_bbox, resize_image, safe_as_tensor
  from pytorch3d.implicitron.tools.config import registry, ReplaceableBase
  from pytorch3d.renderer.camera_utils import join_cameras_as_batch
  from pytorch3d.renderer.cameras import CamerasBase, PerspectiveCameras
  from pytorch3d.structures.pointclouds import join_pointclouds_as_batch, Pointclouds
label_desc: Use image node_id 2 for calling the GenericFrameDataBuilder obj's underlying member method code with example usage: obj.build(frame_annotation, sequence_annotation) and returns: frame_data