class_docstr: RCNN: Region-based Convolutional Neural Network.
  
  Given region proposals (bounding boxes on an image) and a feature map of
  that image, RCNN adjusts the bounding boxes and classifies each region as
  either background or a specific object class.
  
  Steps:
      1. Region of Interest Pooling. Extract features from the feature map
         (based on the proposals) and convert into fixed size tensors
         (applying extrapolation).
      2. Two fully connected layers generate a smaller tensor for each
         region.
      3. A fully conected layer outputs the probability distribution over the
         classes (plus a background class), and another fully connected layer
         outputs the bounding box regressions (one 4-d regression for each of
         the possible classes).
  
  Using the class probability, filter regions classified as background. For
  the remaining regions, use the class probability together with the
  corresponding bounding box regression offsets to generate the final object
  bounding boxes, with classes and probabilities assigned.
class_docstr_tok: ["RCNN",":","Region-based","Convolutional","Neural","Network",".","Given","region","proposals","(","bounding","boxes","on","an","image",")","and","a","feature","map","of","that","image",",","RCNN","adjusts","the","bounding","boxes","and","classifies","each","region","as","either","background","or","a","specific","object","class",".","Steps",":","1",".","Region","of","Interest","Pooling",".","Extract","features","from","the","feature","map","(","based","on","the","proposals",")","and","convert","into","fixed","size","tensors","(","applying","extrapolation",")",".","2",".","Two","fully","connected","layers","generate","a","smaller","tensor","for","each","region",".","3",".","A","fully","conected","layer","outputs","the","probability","distribution","over","the","classes","(","plus","a","background","class",")",",","and","another","fully","connected","layer","outputs","the","bounding","box","regressions","(","one","4-d","regression","for","each","of","the","possible","classes",")",".","Using","the","class","probability",",","filter","regions","classified","as","background",".","For","the","remaining","regions",",","use","the","class","probability","together","with","the","corresponding","bounding","box","regression","offsets","to","generate","the","final","object","bounding","boxes",",","with","classes","and","probabilities","assigned","."]
docstr: Classifies & refines proposals based on the pooled feature map.
  
  Args:
      conv_feature_map: The feature map of the image, extracted
          using the pretrained network.
          Shape: (num_proposals, pool_height, pool_width, 512).
      proposals: A Tensor with the bounding boxes proposed by the RPN.
          Shape: (total_num_proposals, 4).
          Encoding: (x1, y1, x2, y2).
      im_shape: A Tensor with the shape of the image in the form of
          (image_height, image_width).
      gt_boxes (optional): A Tensor with the ground truth boxes of the
          image.
          Shape: (total_num_gt, 5).
          Encoding: (x1, y1, x2, y2, label).
      is_training (optional): A boolean to determine if we are just using
          the module for training or just inference.
  
  Returns:
      prediction_dict: a dict with the object predictions.
          It should have the keys:
          objects:
          labels:
          probs:
  
          rcnn:
          target:
docstr_tok: ["Classifies","&","refines","proposals","based","on","the","pooled","feature","map",".","Args",":","conv_feature_map",":","The","feature","map","of","the","image",",","extracted","using","the","pretrained","network",".","Shape",":","(","num_proposals",",","pool_height",",","pool_width",",","512",")",".","proposals",":","A","Tensor","with","the","bounding","boxes","proposed","by","the","RPN",".","Shape",":","(","total_num_proposals",",","4",")",".","Encoding",":","(","x1",",","y1",",","x2",",","y2",")",".","im_shape",":","A","Tensor","with","the","shape","of","the","image","in","the","form","of","(","image_height",",","image_width",")",".","gt_boxes","(","optional",")",":","A","Tensor","with","the","ground","truth","boxes","of","the","image",".","Shape",":","(","total_num_gt",",","5",")",".","Encoding",":","(","x1",",","y1",",","x2",",","y2",",","label",")",".","is_training","(","optional",")",":","A","boolean","to","determine","if","we","are","just","using","the","module","for","training","or","just","inference",".","Returns",":","prediction_dict",":","a","dict","with","the","object","predictions",".","It","should","have","the","keys",":","objects",":","labels",":","probs",":","rcnn",":","target",":"]
code: def _build(
      self,
      conv_feature_map,
      proposals,
      im_shape,
      base_network,
      gt_boxes=None,
      is_training=False,
  ):
      """
      Classifies & refines proposals based on the pooled feature map.
  
      Args:
          conv_feature_map: The feature map of the image, extracted
              using the pretrained network.
              Shape: (num_proposals, pool_height, pool_width, 512).
          proposals: A Tensor with the bounding boxes proposed by the RPN.
              Shape: (total_num_proposals, 4).
              Encoding: (x1, y1, x2, y2).
          im_shape: A Tensor with the shape of the image in the form of
              (image_height, image_width).
          gt_boxes (optional): A Tensor with the ground truth boxes of the
              image.
              Shape: (total_num_gt, 5).
              Encoding: (x1, y1, x2, y2, label).
          is_training (optional): A boolean to determine if we are just using
              the module for training or just inference.
  
      Returns:
          prediction_dict: a dict with the object predictions.
              It should have the keys:
              objects:
              labels:
              probs:
  
              rcnn:
              target:
  
      """
      self._instantiate_layers()
  
      prediction_dict = {"_debug": {}}
  
      if gt_boxes is not None:
          proposals_target, bbox_offsets_target = self._rcnn_target(
              proposals, gt_boxes
          )
  
          if is_training:
              with tf.name_scope("prepare_batch"):
                  # We flatten to set shape, but it is already a flat Tensor.
                  in_batch_proposals = tf.reshape(
                      tf.greater_equal(proposals_target, 0), [-1]
                  )
                  proposals = tf.boolean_mask(
                      proposals, in_batch_proposals
                  )
                  bbox_offsets_target = tf.boolean_mask(
                      bbox_offsets_target, in_batch_proposals
                  )
                  proposals_target = tf.boolean_mask(
                      proposals_target, in_batch_proposals
                  )
  
          prediction_dict["target"] = {
              "cls": proposals_target,
              "bbox_offsets": bbox_offsets_target,
          }
  
      roi_prediction = self._roi_pool(
          proposals, conv_feature_map, im_shape
      )
  
      if self._debug:
          # Save raw roi prediction in debug mode.
          prediction_dict["_debug"]["roi"] = roi_prediction
  
      pooled_features = roi_prediction["roi_pool"]
      features = base_network._build_tail(
          pooled_features, is_training=is_training
      )
  
      if self._use_mean:
          # We avg our height and width dimensions for a more
          # "memory-friendly" Tensor.
          features = tf.reduce_mean(features, [1, 2])
  
      # We treat num proposals as batch number so that when flattening we
      # get a (num_proposals, flatten_pooled_feature_map_size) Tensor.
      flatten_features = tf.contrib.layers.flatten(features)
      net = tf.identity(flatten_features)
  
      if is_training:
          net = tf.nn.dropout(net, keep_prob=self._dropout_keep_prob)
  
      if self._debug:
          prediction_dict["_debug"]["flatten_net"] = net
  
      # After flattening we are left with a Tensor of shape
      # (num_proposals, pool_height * pool_width * 512).
      # The first dimension works as batch size when applied to snt.Linear.
      for i, layer in enumerate(self._layers):
          # Through FC layer.
          net = layer(net)
  
          # Apply activation and dropout.
          variable_summaries(
              net, "fc_{}_preactivationout".format(i), "reduced"
          )
          net = self._activation(net)
          if self._debug:
              prediction_dict["_debug"]["layer_{}_out".format(i)] = net
  
          variable_summaries(net, "fc_{}_out".format(i), "reduced")
          if is_training:
              net = tf.nn.dropout(
                  net, keep_prob=self._dropout_keep_prob
              )
  
      cls_score = self._classifier_layer(net)
      cls_prob = tf.nn.softmax(cls_score, axis=1)
      bbox_offsets = self._bbox_layer(net)
  
      prediction_dict["rcnn"] = {
          "cls_score": cls_score,
          "cls_prob": cls_prob,
          "bbox_offsets": bbox_offsets,
      }
  
      # Get final objects proposals based on the probabilty, the offsets and
      # the original proposals.
      proposals_pred = self._rcnn_proposal(
          proposals, bbox_offsets, cls_prob, im_shape
      )
  
      # objects, objects_labels, and objects_labels_prob are the only keys
      # that matter for drawing objects.
      prediction_dict["objects"] = proposals_pred["objects"]
      prediction_dict["labels"] = proposals_pred["proposal_label"]
      prediction_dict["probs"] = proposals_pred["proposal_label_prob"]
  
      if self._debug:
          prediction_dict["_debug"]["proposal"] = proposals_pred
  
      # Calculate summaries for results
      variable_summaries(cls_prob, "cls_prob", "reduced")
      variable_summaries(bbox_offsets, "bbox_offsets", "reduced")
  
      if self._debug:
          variable_summaries(pooled_features, "pooled_features", "full")
          layer_summaries(self._classifier_layer, "full")
          layer_summaries(self._bbox_layer, "full")
  
      return prediction_dict
  
file_path: luminoth/luminoth/models/fasterrcnn/rcnn.py
imports: import sonnet
  import tensorflow
  from luminoth.models.fasterrcnn.rcnn_proposal import RCNNProposal
  from luminoth.models.fasterrcnn.rcnn_target import RCNNTarget
  from luminoth.models.fasterrcnn.roi_pool import ROIPoolingLayer
  from luminoth.utils.losses import smooth_l1_loss
  from luminoth.utils.vars import get_initializer, layer_summaries, variable_summaries, get_activation_function
label_desc: Use image node_id 3 for calling the RCNN obj's underlying member method code with example usage: obj._build(conv_feature_map, proposals, im_shape, base_network, gt_boxes, is_training) and returns: prediction_dict