class_docstr: Differentiable rendering module for the Pulsar renderer.
  
  Set the maximum number of balls to a reasonable value. It is used to determine
  several buffer sizes. It is no problem to render less balls than this number,
  but never more.
  
  When optimizing for sphere positions, sphere radiuses or camera parameters you
  have to use higher gamma values (closer to one) and larger sphere sizes: spheres
  can only 'move' to areas that they cover, and only with higher gamma values exists
  a gradient w.r.t. their color depending on their position.
  
  Args:
      * width: result image width in pixels.
      * height: result image height in pixels.
      * max_num_balls: the maximum number of balls this renderer will handle.
      * orthogonal_projection: use an orthogonal instead of perspective projection.
          Default: False.
      * right_handed_system: use a right-handed instead of a left-handed coordinate
          system. This is relevant for compatibility with other drawing or scanning
          systems. Pulsar by default assumes a left-handed world and camera coordinate
          system as known from mathematics with x-axis to the right, y axis up and z
          axis for increasing depth along the optical axis. In the image coordinate
          system, only the y axis is pointing down, leading still to a left-handed
          system. If you set this to True, it is assuming a right-handed world and
          camera coordinate system with x axis to the right, y axis to the top and
          z axis decreasing along the optical axis. Again, the image coordinate
          system has a flipped y axis, remaining a right-handed system.
          Default: False.
      * background_normalized_depth: the normalized depth the background is placed
          at.
          This is on a scale from 0. to 1. between the specified min and max depth
          (see the forward function). The value 0. is the most furthest depth whereas
          1. is the closest. Be careful when setting the background too far front - it
          may hide elements in your scene. Default: EPS.
      * n_channels: the number of image content channels to use. This is usually three
          for regular color representations, but can be a higher or lower number.
          Default: 3.
      * n_track: the number of spheres to track for gradient calculation per pixel.
          Only the closest n_track spheres will receive gradients. Default: 5.
class_docstr_tok: ["Differentiable","rendering","module","for","the","Pulsar","renderer",".","Set","the","maximum","number","of","balls","to","a","reasonable","value",".","It","is","used","to","determine","several","buffer","sizes",".","It","is","no","problem","to","render","less","balls","than","this","number",",","but","never","more",".","When","optimizing","for","sphere","positions",",","sphere","radiuses","or","camera","parameters","you","have","to","use","higher","gamma","values","(","closer","to","one",")","and","larger","sphere","sizes",":","spheres","can","only","'move","'","to","areas","that","they","cover",",","and","only","with","higher","gamma","values","exists","a","gradient","w.r.t",".","their","color","depending","on","their","position",".","Args",":","*","width",":","result","image","width","in","pixels",".","*","height",":","result","image","height","in","pixels",".","*","max_num_balls",":","the","maximum","number","of","balls","this","renderer","will","handle",".","*","orthogonal_projection",":","use","an","orthogonal","instead","of","perspective","projection",".","Default",":","False",".","*","right_handed_system",":","use","a","right-handed","instead","of","a","left-handed","coordinate","system",".","This","is","relevant","for","compatibility","with","other","drawing","or","scanning","systems",".","Pulsar","by","default","assumes","a","left-handed","world","and","camera","coordinate","system","as","known","from","mathematics","with","x-axis","to","the","right",",","y","axis","up","and","z","axis","for","increasing","depth","along","the","optical","axis",".","In","the","image","coordinate","system",",","only","the","y","axis","is","pointing","down",",","leading","still","to","a","left-handed","system",".","If","you","set","this","to","True",",","it","is","assuming","a","right-handed","world","and","camera","coordinate","system","with","x","axis","to","the","right",",","y","axis","to","the","top","and","z","axis","decreasing","along","the","optical","axis",".","Again",",","the","image","coordinate","system","has","a","flipped","y","axis",",","remaining","a","right-handed","system",".","Default",":","False",".","*","background_normalized_depth",":","the","normalized","depth","the","background","is","placed","at",".","This","is","on","a","scale","from","0.","to","1.","between","the","specified","min","and","max","depth","(","see","the","forward","function",")",".","The","value","0.","is","the","most","furthest","depth","whereas","1.","is","the","closest",".","Be","careful","when","setting","the","background","too","far","front","-","it","may","hide","elements","in","your","scene",".","Default",":","EPS",".","*","n_channels",":","the","number","of","image","content","channels","to","use",".","This","is","usually","three","for","regular","color","representations",",","but","can","be","a","higher","or","lower","number",".","Default",":","3",".","*","n_track",":","the","number","of","spheres","to","track","for","gradient","calculation","per","pixel",".","Only","the","closest","n_track","spheres","will","receive","gradients",".","Default",":","5","."]
docstr: Rendering pass to create an image from the provided spheres and camera
  parameters.
  
  Args:
      * vert_pos: vertex positions. [Bx]Nx3 tensor of positions in 3D space.
      * vert_col: vertex colors. [Bx]NxK tensor of channels.
      * vert_rad: vertex radii. [Bx]N tensor of radiuses, >0.
      * cam_params: camera parameter(s). [Bx]8 tensor, consisting of:
          - 3 components for camera position,
          - 3 components for camera rotation (axis angle representation) or
            6 components as described in "On the Continuity of Rotation
            Representations in Neural Networks" (Zhou et al.),
          - focal length,
          - the sensor width in world coordinates,
          - [optional] an offset for the principal point in x, y (no gradients).
      * gamma: sphere transparency in [1.,1E-5], with 1 being mostly transparent.
          [Bx]1.
      * max_depth: maximum depth for spheres to render. Set this as tightly
                  as possible to have good numerical accuracy for gradients.
                  float > min_depth + eps.
      * min_depth: a float with the minimum depth a sphere must have to be
                  rendered. Must be 0. or > max(focal_length) + eps.
      * bg_col: K tensor with a background color to use or None (uses all ones).
      * opacity: [Bx]N tensor of opacity values in [0., 1.] or None (uses all
              ones).
      * percent_allowed_difference: a float in [0., 1.[ with the maximum allowed
                  difference in color space. This is used to speed up the
                  computation. Default: 0.01.
      * max_n_hits: a hard limit on the number of hits per ray. Default: max int.
      * mode: render mode in {0, 1}. 0: render an image; 1: render the hit map.
      * return_forward_info: whether to return a second map. This second map
          contains 13 channels: first channel contains sm_m (the maximum
          exponent factor observed), the second sm_d (the normalization
          denominator, the sum of all coefficients), the third the maximum closest
          possible intersection for a hit. The following channels alternate with
          the float encoded integer index of a sphere and its weight. They are the
          five spheres with the highest color contribution to this pixel color,
          ordered descending. Default: False.
      * first_R_then_T: bool, whether to first apply rotation to the camera,
          then translation (PyTorch3D convention). Default: False.
  
  Returns:
      * image: [Bx]HxWx3 float tensor with the resulting image.
      * forw_info: [Bx]HxWx13 float forward information as described above, if
              enabled.
docstr_tok: ["Rendering","pass","to","create","an","image","from","the","provided","spheres","and","camera","parameters",".","Args",":","*","vert_pos",":","vertex","positions",".","[","Bx","]","Nx3","tensor","of","positions","in","3D","space",".","*","vert_col",":","vertex","colors",".","[","Bx","]","NxK","tensor","of","channels",".","*","vert_rad",":","vertex","radii",".","[","Bx","]","N","tensor","of","radiuses",",",">","0",".","*","cam_params",":","camera","parameter","(","s",")",".","[","Bx","]","8","tensor",",","consisting","of",":","-","3","components","for","camera","position",",","-","3","components","for","camera","rotation","(","axis","angle","representation",")","or","6","components","as","described","in","``","On","the","Continuity","of","Rotation","Representations","in","Neural","Networks","''","(","Zhou","et","al",".",")",",","-","focal","length",",","-","the","sensor","width","in","world","coordinates",",","-","[","optional","]","an","offset","for","the","principal","point","in","x",",","y","(","no","gradients",")",".","*","gamma",":","sphere","transparency","in","[","1.,1E-5","]",",","with","1","being","mostly","transparent",".","[","Bx","]","1",".","*","max_depth",":","maximum","depth","for","spheres","to","render",".","Set","this","as","tightly","as","possible","to","have","good","numerical","accuracy","for","gradients",".","float",">","min_depth","+","eps",".","*","min_depth",":","a","float","with","the","minimum","depth","a","sphere","must","have","to","be","rendered",".","Must","be","0.","or",">","max","(","focal_length",")","+","eps",".","*","bg_col",":","K","tensor","with","a","background","color","to","use","or","None","(","uses","all","ones",")",".","*","opacity",":","[","Bx","]","N","tensor","of","opacity","values","in","[","0.",",","1",".","]","or","None","(","uses","all","ones",")",".","*","percent_allowed_difference",":","a","float","in","[","0.",",","1",".","[","with","the","maximum","allowed","difference","in","color","space",".","This","is","used","to","speed","up","the","computation",".","Default",":","0.01",".","*","max_n_hits",":","a","hard","limit","on","the","number","of","hits","per","ray",".","Default",":","max","int",".","*","mode",":","render","mode","in","{","0",",","1","}",".","0",":","render","an","image",";","1",":","render","the","hit","map",".","*","return_forward_info",":","whether","to","return","a","second","map",".","This","second","map","contains","13","channels",":","first","channel","contains","sm_m","(","the","maximum","exponent","factor","observed",")",",","the","second","sm_d","(","the","normalization","denominator",",","the","sum","of","all","coefficients",")",",","the","third","the","maximum","closest","possible","intersection","for","a","hit",".","The","following","channels","alternate","with","the","float","encoded","integer","index","of","a","sphere","and","its","weight",".","They","are","the","five","spheres","with","the","highest","color","contribution","to","this","pixel","color",",","ordered","descending",".","Default",":","False",".","*","first_R_then_T",":","bool",",","whether","to","first","apply","rotation","to","the","camera",",","then","translation","(","PyTorch3D","convention",")",".","Default",":","False",".","Returns",":","*","image",":","[","Bx","]","HxWx3","float","tensor","with","the","resulting","image",".","*","forw_info",":","[","Bx","]","HxWx13","float","forward","information","as","described","above",",","if","enabled","."]
code: def forward(
      self,
      vert_pos: torch.Tensor,
      vert_col: torch.Tensor,
      vert_rad: torch.Tensor,
      cam_params: torch.Tensor,
      gamma: float,
      max_depth: float,
      min_depth: float = 0.0,
      bg_col: Optional[torch.Tensor] = None,
      opacity: Optional[torch.Tensor] = None,
      percent_allowed_difference: float = 0.01,
      # pyre-fixme[16]: Module `_C` has no attribute `MAX_UINT`.
      max_n_hits: int = _C.MAX_UINT,
      mode: int = 0,
      return_forward_info: bool = False,
      first_R_then_T: bool = False,
  ) -> Union[torch.Tensor, Tuple[torch.Tensor, Optional[torch.Tensor]]]:
      """
      Rendering pass to create an image from the provided spheres and camera
      parameters.
  
      Args:
          * vert_pos: vertex positions. [Bx]Nx3 tensor of positions in 3D space.
          * vert_col: vertex colors. [Bx]NxK tensor of channels.
          * vert_rad: vertex radii. [Bx]N tensor of radiuses, >0.
          * cam_params: camera parameter(s). [Bx]8 tensor, consisting of:
              - 3 components for camera position,
              - 3 components for camera rotation (axis angle representation) or
                6 components as described in "On the Continuity of Rotation
                Representations in Neural Networks" (Zhou et al.),
              - focal length,
              - the sensor width in world coordinates,
              - [optional] an offset for the principal point in x, y (no gradients).
          * gamma: sphere transparency in [1.,1E-5], with 1 being mostly transparent.
              [Bx]1.
          * max_depth: maximum depth for spheres to render. Set this as tightly
                      as possible to have good numerical accuracy for gradients.
                      float > min_depth + eps.
          * min_depth: a float with the minimum depth a sphere must have to be
                      rendered. Must be 0. or > max(focal_length) + eps.
          * bg_col: K tensor with a background color to use or None (uses all ones).
          * opacity: [Bx]N tensor of opacity values in [0., 1.] or None (uses all
                  ones).
          * percent_allowed_difference: a float in [0., 1.[ with the maximum allowed
                      difference in color space. This is used to speed up the
                      computation. Default: 0.01.
          * max_n_hits: a hard limit on the number of hits per ray. Default: max int.
          * mode: render mode in {0, 1}. 0: render an image; 1: render the hit map.
          * return_forward_info: whether to return a second map. This second map
              contains 13 channels: first channel contains sm_m (the maximum
              exponent factor observed), the second sm_d (the normalization
              denominator, the sum of all coefficients), the third the maximum closest
              possible intersection for a hit. The following channels alternate with
              the float encoded integer index of a sphere and its weight. They are the
              five spheres with the highest color contribution to this pixel color,
              ordered descending. Default: False.
          * first_R_then_T: bool, whether to first apply rotation to the camera,
              then translation (PyTorch3D convention). Default: False.
  
      Returns:
          * image: [Bx]HxWx3 float tensor with the resulting image.
          * forw_info: [Bx]HxWx13 float forward information as described above, if
                  enabled.
      """
      # The device tracker is registered as buffer.
      self._renderer.device_tracker = self.device_tracker
      (
          pos_vec,
          pixel_0_0_center,
          pixel_vec_x,
          pixel_vec_y,
          focal_lengths,
          principal_point_offsets,
      ) = Renderer._transform_cam_params(
          cam_params,
          self._renderer.width,
          self._renderer.height,
          self._renderer.orthogonal,
          self._renderer.right_handed,
          first_R_then_T=first_R_then_T,
      )
      if (
          focal_lengths.min().item() > 0.0
          and max_depth > 10_000.0 * focal_lengths.min().item()
      ):
          warnings.warn(
              (
                  "Extreme ratio of `max_depth` vs. focal length detected "
                  "(%f vs. %f, ratio: %f). This will likely lead to "
                  "artifacts due to numerical instabilities."
              )
              % (
                  max_depth,
                  focal_lengths.min().item(),
                  max_depth / focal_lengths.min().item(),
              )
          )
      ret_res = _Render.apply(
          vert_pos,
          vert_col,
          vert_rad,
          pos_vec,
          pixel_0_0_center,
          pixel_vec_x,
          pixel_vec_y,
          # Focal length and sensor size don't need gradients other than through
          # `pixel_vec_x` and `pixel_vec_y`. The focal length is only used in the
          # renderer to determine the projection areas of the balls.
          focal_lengths,
          # principal_point_offsets does not receive gradients.
          principal_point_offsets,
          gamma,
          max_depth,
          self._renderer,
          min_depth,
          bg_col,
          opacity,
          percent_allowed_difference,
          max_n_hits,
          mode,
          (mode == 0) and return_forward_info,
      )
      if return_forward_info and mode != 0:
          return ret_res, None
      return ret_res
  
file_path: pytorch3d/pytorch3d/renderer/points/pulsar/renderer.py
imports: import logging
  import warnings
  from typing import Optional, Tuple, Union
  import torch
  from pytorch3d import _C
  from pytorch3d.transforms import axis_angle_to_matrix, rotation_6d_to_matrix
label_desc: Use image node_id 5 for calling the Renderer obj's underlying member method code with example usage: obj.forward(vert_pos, vert_col, vert_rad, cam_params, gamma, max_depth, min_depth, bg_col, opacity, percent_allowed_difference, max_n_hits, mode, return_forward_info, first_R_then_T) and returns: ret_res, ret_res,  None