class_docstr: Double Head RCNN network.
  
  Parameters
  ----------
  features : gluon.HybridBlock
      Base feature extractor before feature pooling layer.
  top_features : gluon.HybridBlock
      Tail feature extractor after feature pooling layer.
  classes : iterable of str
      Names of categories, its length is ``num_class``.
  box_features : gluon.HybridBlock, default is None
      feature head for transforming shared ROI output (top_features) for box prediction.
      If set to None, global average pooling will be used.
  short : int, default is 600.
      Input image short side size.
  max_size : int, default is 1000.
      Maximum size of input image long side.
  min_stage : int, default is 4
      Minimum stage NO. for FPN stages.
  max_stage : int, default is 4
      Maximum stage NO. for FPN stages.
  train_patterns : str, default is None.
      Matching pattern for trainable parameters.
  nms_thresh : float, default is 0.3.
      Non-maximum suppression threshold. You can specify < 0 or > 1 to disable NMS.
  nms_topk : int, default is 400
      Apply NMS to top k detection results, use -1 to disable so that every Detection
      result is used in NMS.
  roi_mode : str, default is align
      ROI pooling mode. Currently support 'pool' and 'align'.
  roi_size : tuple of int, length 2, default is (14, 14)
      (height, width) of the ROI region.
  strides : int/tuple of ints, default is 16
      Feature map stride with respect to original image.
      This is usually the ratio between original image size and feature map size.
      For FPN, use a tuple of ints.
  clip : float, default is None
      Clip bounding box prediction to to prevent exponentiation from overflowing.
  rpn_channel : int, default is 1024
      number of channels used in RPN convolutional layers.
  base_size : int
      The width(and height) of reference anchor box.
  scales : iterable of float, default is (8, 16, 32)
      The areas of anchor boxes.
      We use the following form to compute the shapes of anchors:
  
      .. math::
  
          width_{anchor} = size_{base} \times scale \times \sqrt{ 1 / ratio}
          height_{anchor} = size_{base} \times scale \times \sqrt{ratio}
  
  ratios : iterable of float, default is (0.5, 1, 2)
      The aspect ratios of anchor boxes. We expect it to be a list or tuple.
  alloc_size : tuple of int
      Allocate size for the anchor boxes as (H, W).
      Usually we generate enough anchors for large feature map, e.g. 128x128.
      Later in inference we can have variable input sizes,
      at which time we can crop corresponding anchors from this large
      anchor map so we can skip re-generating anchors for each input.
  rpn_train_pre_nms : int, default is 12000
      Filter top proposals before NMS in training of RPN.
  rpn_train_post_nms : int, default is 2000
      Return top proposal results after NMS in training of RPN.
      Will be set to rpn_train_pre_nms if it is larger than rpn_train_pre_nms.
  rpn_test_pre_nms : int, default is 6000
      Filter top proposals before NMS in testing of RPN.
  rpn_test_post_nms : int, default is 300
      Return top proposal results after NMS in testing of RPN.
      Will be set to rpn_test_pre_nms if it is larger than rpn_test_pre_nms.
  rpn_nms_thresh : float, default is 0.7
      IOU threshold for NMS. It is used to remove overlapping proposals.
  rpn_num_sample : int, default is 256
      Number of samples for RPN targets.
  rpn_pos_iou_thresh : float, default is 0.7
      Anchor with IOU larger than ``pos_iou_thresh`` is regarded as positive samples.
  rpn_neg_iou_thresh : float, default is 0.3
      Anchor with IOU smaller than ``neg_iou_thresh`` is regarded as negative samples.
      Anchors with IOU in between ``pos_iou_thresh`` and ``neg_iou_thresh`` are
      ignored.
  rpn_pos_ratio : float, default is 0.5
      ``pos_ratio`` defines how many positive samples (``pos_ratio * num_sample``) is
      to be sampled.
  rpn_box_norm : array-like of size 4, default is (1., 1., 1., 1.)
      Std value to be divided from encoded values.
  rpn_min_size : int, default is 16
      Proposals whose size is smaller than ``min_size`` will be discarded.
  per_device_batch_size : int, default is 1
      Batch size for each device during training.
  num_sample : int, default is 128
      Number of samples for RCNN targets.
  pos_iou_thresh : float, default is 0.5
      Proposal whose IOU larger than ``pos_iou_thresh`` is regarded as positive samples.
  pos_ratio : float, default is 0.25
      ``pos_ratio`` defines how many positive samples (``pos_ratio * num_sample``) is
      to be sampled.
  max_num_gt : int, default is 300
      Maximum ground-truth number for each example. This is only an upper bound, not
      necessarily very precise. However, using a very big number may impact the training speed.
  additional_output : boolean, default is False
      ``additional_output`` is only used for Mask R-CNN to get internal outputs.
  force_nms : bool, default is False
      Appy NMS to all categories, this is to avoid overlapping detection results from different
      categories.
  minimal_opset : bool, default is `False`
      We sometimes add special operators to accelerate training/inference, however, for exporting
      to third party compilers we want to utilize most widely used operators.
      If `minimal_opset` is `True`, the network will use a minimal set of operators good
      for e.g., `TVM`.
  
  Attributes
  ----------
  classes : iterable of str
      Names of categories, its length is ``num_class``.
  num_class : int
      Number of positive categories.
  short : int
      Input image short side size.
  max_size : int
      Maximum size of input image long side.
  train_patterns : str
      Matching pattern for trainable parameters.
  nms_thresh : float
      Non-maximum suppression threshold. You can specify < 0 or > 1 to disable NMS.
  nms_topk : int
      Apply NMS to top k detection results, use -1 to disable so that every Detection
       result is used in NMS.
  force_nms : bool
      Appy NMS to all categories, this is to avoid overlapping detection results
      from different categories.
  rpn_target_generator : gluon.Block
      Generate training targets with cls_target, box_target, and box_mask.
  target_generator : gluon.Block
      Generate training targets with boxes, samples, matches, gt_label and gt_box.
class_docstr_tok: ["Double","Head","RCNN","network",".","Parameters","--","--","--","--","--","features",":","gluon.HybridBlock","Base","feature","extractor","before","feature","pooling","layer",".","top_features",":","gluon.HybridBlock","Tail","feature","extractor","after","feature","pooling","layer",".","classes",":","iterable","of","str","Names","of","categories",",","its","length","is","``","num_class","``",".","box_features",":","gluon.HybridBlock",",","default","is","None","feature","head","for","transforming","shared","ROI","output","(","top_features",")","for","box","prediction",".","If","set","to","None",",","global","average","pooling","will","be","used",".","short",":","int",",","default","is","600",".","Input","image","short","side","size",".","max_size",":","int",",","default","is","1000",".","Maximum","size","of","input","image","long","side",".","min_stage",":","int",",","default","is","4","Minimum","stage","NO",".","for","FPN","stages",".","max_stage",":","int",",","default","is","4","Maximum","stage","NO",".","for","FPN","stages",".","train_patterns",":","str",",","default","is","None",".","Matching","pattern","for","trainable","parameters",".","nms_thresh",":","float",",","default","is","0.3",".","Non-maximum","suppression","threshold",".","You","can","specify","<","0","or",">","1","to","disable","NMS",".","nms_topk",":","int",",","default","is","400","Apply","NMS","to","top","k","detection","results",",","use","-1","to","disable","so","that","every","Detection","result","is","used","in","NMS",".","roi_mode",":","str",",","default","is","align","ROI","pooling","mode",".","Currently","support","'pool","'","and","'align","'",".","roi_size",":","tuple","of","int",",","length","2",",","default","is","(","14",",","14",")","(","height",",","width",")","of","the","ROI","region",".","strides",":","int\/tuple","of","ints",",","default","is","16","Feature","map","stride","with","respect","to","original","image",".","This","is","usually","the","ratio","between","original","image","size","and","feature","map","size",".","For","FPN",",","use","a","tuple","of","ints",".","clip",":","float",",","default","is","None","Clip","bounding","box","prediction","to","to","prevent","exponentiation","from","overflowing",".","rpn_channel",":","int",",","default","is","1024","number","of","channels","used","in","RPN","convolutional","layers",".","base_size",":","int","The","width","(","and","height",")","of","reference","anchor","box",".","scales",":","iterable","of","float",",","default","is","(","8",",","16",",","32",")","The","areas","of","anchor","boxes",".","We","use","the","following","form","to","compute","the","shapes","of","anchors",":","..","math",":",":","width_","{","anchor","}","=","size_","{","base","}","\\times","scale","\\times","\\sqrt","{","1","\/","ratio","}","height_","{","anchor","}","=","size_","{","base","}","\\times","scale","\\times","\\sqrt","{","ratio","}","ratios",":","iterable","of","float",",","default","is","(","0.5",",","1",",","2",")","The","aspect","ratios","of","anchor","boxes",".","We","expect","it","to","be","a","list","or","tuple",".","alloc_size",":","tuple","of","int","Allocate","size","for","the","anchor","boxes","as","(","H",",","W",")",".","Usually","we","generate","enough","anchors","for","large","feature","map",",","e.g",".","128x128",".","Later","in","inference","we","can","have","variable","input","sizes",",","at","which","time","we","can","crop","corresponding","anchors","from","this","large","anchor","map","so","we","can","skip","re-generating","anchors","for","each","input",".","rpn_train_pre_nms",":","int",",","default","is","12000","Filter","top","proposals","before","NMS","in","training","of","RPN",".","rpn_train_post_nms",":","int",",","default","is","2000","Return","top","proposal","results","after","NMS","in","training","of","RPN",".","Will","be","set","to","rpn_train_pre_nms","if","it","is","larger","than","rpn_train_pre_nms",".","rpn_test_pre_nms",":","int",",","default","is","6000","Filter","top","proposals","before","NMS","in","testing","of","RPN",".","rpn_test_post_nms",":","int",",","default","is","300","Return","top","proposal","results","after","NMS","in","testing","of","RPN",".","Will","be","set","to","rpn_test_pre_nms","if","it","is","larger","than","rpn_test_pre_nms",".","rpn_nms_thresh",":","float",",","default","is","0.7","IOU","threshold","for","NMS",".","It","is","used","to","remove","overlapping","proposals",".","rpn_num_sample",":","int",",","default","is","256","Number","of","samples","for","RPN","targets",".","rpn_pos_iou_thresh",":","float",",","default","is","0.7","Anchor","with","IOU","larger","than","``","pos_iou_thresh","``","is","regarded","as","positive","samples",".","rpn_neg_iou_thresh",":","float",",","default","is","0.3","Anchor","with","IOU","smaller","than","``","neg_iou_thresh","``","is","regarded","as","negative","samples",".","Anchors","with","IOU","in","between","``","pos_iou_thresh","``","and","``","neg_iou_thresh","``","are","ignored",".","rpn_pos_ratio",":","float",",","default","is","0.5","``","pos_ratio","``","defines","how","many","positive","samples","(","``","pos_ratio","*","num_sample","``",")","is","to","be","sampled",".","rpn_box_norm",":","array-like","of","size","4",",","default","is","(","1.",",","1.",",","1.",",","1",".",")","Std","value","to","be","divided","from","encoded","values",".","rpn_min_size",":","int",",","default","is","16","Proposals","whose","size","is","smaller","than","``","min_size","``","will","be","discarded",".","per_device_batch_size",":","int",",","default","is","1","Batch","size","for","each","device","during","training",".","num_sample",":","int",",","default","is","128","Number","of","samples","for","RCNN","targets",".","pos_iou_thresh",":","float",",","default","is","0.5","Proposal","whose","IOU","larger","than","``","pos_iou_thresh","``","is","regarded","as","positive","samples",".","pos_ratio",":","float",",","default","is","0.25","``","pos_ratio","``","defines","how","many","positive","samples","(","``","pos_ratio","*","num_sample","``",")","is","to","be","sampled",".","max_num_gt",":","int",",","default","is","300","Maximum","ground-truth","number","for","each","example",".","This","is","only","an","upper","bound",",","not","necessarily","very","precise",".","However",",","using","a","very","big","number","may","impact","the","training","speed",".","additional_output",":","boolean",",","default","is","False","``","additional_output","``","is","only","used","for","Mask","R-CNN","to","get","internal","outputs",".","force_nms",":","bool",",","default","is","False","Appy","NMS","to","all","categories",",","this","is","to","avoid","overlapping","detection","results","from","different","categories",".","minimal_opset",":","bool",",","default","is","`","False","`","We","sometimes","add","special","operators","to","accelerate","training\/inference",",","however",",","for","exporting","to","third","party","compilers","we","want","to","utilize","most","widely","used","operators",".","If","`","minimal_opset","`","is","`","True","`",",","the","network","will","use","a","minimal","set","of","operators","good","for","e.g.",",","`","TVM","`",".","Attributes","--","--","--","--","--","classes",":","iterable","of","str","Names","of","categories",",","its","length","is","``","num_class","``",".","num_class",":","int","Number","of","positive","categories",".","short",":","int","Input","image","short","side","size",".","max_size",":","int","Maximum","size","of","input","image","long","side",".","train_patterns",":","str","Matching","pattern","for","trainable","parameters",".","nms_thresh",":","float","Non-maximum","suppression","threshold",".","You","can","specify","<","0","or",">","1","to","disable","NMS",".","nms_topk",":","int","Apply","NMS","to","top","k","detection","results",",","use","-1","to","disable","so","that","every","Detection","result","is","used","in","NMS",".","force_nms",":","bool","Appy","NMS","to","all","categories",",","this","is","to","avoid","overlapping","detection","results","from","different","categories",".","rpn_target_generator",":","gluon.Block","Generate","training","targets","with","cls_target",",","box_target",",","and","box_mask",".","target_generator",":","gluon.Block","Generate","training","targets","with","boxes",",","samples",",","matches",",","gt_label","and","gt_box","."]
docstr: Assign rpn_rois to specific FPN layers according to its area
     and then perform `ROIPooling` or `ROIAlign` to generate final
     region proposals aggregated features.
  Parameters
  ----------
  features : list of mx.ndarray or mx.symbol
      Features extracted from FPN base network
  rpn_rois : mx.ndarray or mx.symbol
      (N, 5) with [[batch_index, x1, y1, x2, y2], ...] like
  roi_size : tuple
      The size of each roi with regard to ROI-Wise operation
      each region proposal will be roi_size spatial shape.
  strides : tuple e.g. [4, 8, 16, 32]
      Define the gap between each feature in feature map in the original image space.
  roi_mode : str, default is align
      ROI pooling mode. Currently support 'pool' and 'align'.
  roi_canonical_scale : float, default is 224.0
      Hyperparameters for the RoI-to-FPN level mapping heuristic.
  sampling_ratio : int, default is 2
      number of inputs samples to take for each output
      sample. 0 to take samples densely.
  Returns
  -------
  Pooled roi features aggregated according to its roi_level
docstr_tok: ["Assign","rpn_rois","to","specific","FPN","layers","according","to","its","area","and","then","perform","`","ROIPooling","`","or","`","ROIAlign","`","to","generate","final","region","proposals","aggregated","features",".","Parameters","--","--","--","--","--","features",":","list","of","mx.ndarray","or","mx.symbol","Features","extracted","from","FPN","base","network","rpn_rois",":","mx.ndarray","or","mx.symbol","(","N",",","5",")","with","[","[","batch_index",",","x1",",","y1",",","x2",",","y2","]",",","...","]","like","roi_size",":","tuple","The","size","of","each","roi","with","regard","to","ROI-Wise","operation","each","region","proposal","will","be","roi_size","spatial","shape",".","strides",":","tuple","e.g",".","[","4",",","8",",","16",",","32","]","Define","the","gap","between","each","feature","in","feature","map","in","the","original","image","space",".","roi_mode",":","str",",","default","is","align","ROI","pooling","mode",".","Currently","support","'pool","'","and","'align","'",".","roi_canonical_scale",":","float",",","default","is","224.0","Hyperparameters","for","the","RoI-to-FPN","level","mapping","heuristic",".","sampling_ratio",":","int",",","default","is","2","number","of","inputs","samples","to","take","for","each","output","sample",".","0","to","take","samples","densely",".","Returns","--","--","--","-","Pooled","roi","features","aggregated","according","to","its","roi_level"]
code: def _pyramid_roi_feats(
      self,
      F,
      features,
      rpn_rois,
      roi_size,
      strides,
      roi_mode="align",
      roi_canonical_scale=224.0,
      sampling_ratio=2,
      eps=1e-6,
  ):
      """Assign rpn_rois to specific FPN layers according to its area
         and then perform `ROIPooling` or `ROIAlign` to generate final
         region proposals aggregated features.
      Parameters
      ----------
      features : list of mx.ndarray or mx.symbol
          Features extracted from FPN base network
      rpn_rois : mx.ndarray or mx.symbol
          (N, 5) with [[batch_index, x1, y1, x2, y2], ...] like
      roi_size : tuple
          The size of each roi with regard to ROI-Wise operation
          each region proposal will be roi_size spatial shape.
      strides : tuple e.g. [4, 8, 16, 32]
          Define the gap between each feature in feature map in the original image space.
      roi_mode : str, default is align
          ROI pooling mode. Currently support 'pool' and 'align'.
      roi_canonical_scale : float, default is 224.0
          Hyperparameters for the RoI-to-FPN level mapping heuristic.
      sampling_ratio : int, default is 2
          number of inputs samples to take for each output
          sample. 0 to take samples densely.
      Returns
      -------
      Pooled roi features aggregated according to its roi_level
      """
      max_stage = self._max_stage
      if self._max_stage > 5:  # do not use p6 for RCNN
          max_stage = self._max_stage - 1
      _, x1, y1, x2, y2 = F.split(rpn_rois, axis=-1, num_outputs=5)
      h = y2 - y1 + 1
      w = x2 - x1 + 1
      roi_level = F.floor(
          4 + F.log2(F.sqrt(w * h) / roi_canonical_scale + eps)
      )
      roi_level = F.squeeze(
          F.clip(roi_level, self._min_stage, max_stage)
      )
      # [2,2,..,3,3,...,4,4,...,5,5,...] ``Prohibit swap order here``
      # roi_level_sorted_args = F.argsort(roi_level, is_ascend=True)
      # roi_level = F.sort(roi_level, is_ascend=True)
      # rpn_rois = F.take(rpn_rois, roi_level_sorted_args, axis=0)
      pooled_roi_feats = []
      for i, l in enumerate(range(self._min_stage, max_stage + 1)):
          if roi_mode == "pool":
              # Pool features with all rois first, and then set invalid pooled features to zero,
              # at last ele-wise add together to aggregate all features.
              pooled_feature = F.ROIPooling(
                  features[i], rpn_rois, roi_size, 1.0 / strides[i]
              )
              pooled_feature = F.where(
                  roi_level == l,
                  pooled_feature,
                  F.zeros_like(pooled_feature),
              )
          elif roi_mode == "align":
              if (
                  "box_encode" in F.contrib.__dict__
                  and "box_decode" in F.contrib.__dict__
              ):
                  # TODO(jerryzcn): clean this up for once mx 1.6 is released.
                  masked_rpn_rois = F.where(
                      roi_level == l,
                      rpn_rois,
                      F.ones_like(rpn_rois) * -1.0,
                  )
                  pooled_feature = F.contrib.ROIAlign(
                      features[i],
                      masked_rpn_rois,
                      roi_size,
                      1.0 / strides[i],
                      sample_ratio=sampling_ratio,
                  )
              else:
                  pooled_feature = F.contrib.ROIAlign(
                      features[i],
                      rpn_rois,
                      roi_size,
                      1.0 / strides[i],
                      sample_ratio=sampling_ratio,
                  )
                  pooled_feature = F.where(
                      roi_level == l,
                      pooled_feature,
                      F.zeros_like(pooled_feature),
                  )
          else:
              raise ValueError("Invalid roi mode: {}".format(roi_mode))
          pooled_roi_feats.append(pooled_feature)
      # Ele-wise add to aggregate all pooled features
      pooled_roi_feats = F.ElementWiseSum(*pooled_roi_feats)
      # Sort all pooled features by asceding order
      # [2,2,..,3,3,...,4,4,...,5,5,...]
      # pooled_roi_feats = F.take(pooled_roi_feats, roi_level_sorted_args)
      # pooled roi feats (B*N, C, 7, 7), N = N2 + N3 + N4 + N5 = num_roi, C=256 in ori paper
      return pooled_roi_feats
  
imports: from __future__ import absolute_import
  import os
  import mxnet
  from mxnet import autograd
  from mxnet.gluon import nn
  from mxnet.gluon.contrib.nn import SyncBatchNorm
  from mxnet.gluon.nn import BatchNorm
  from mxnet.gluon.block import HybridBlock
  from .rcnn_target import RCNNTargetSampler, RCNNTargetGenerator
  from ..rcnn import custom_rcnn_fpn
  from ....model_zoo.rcnn import RCNN
  from ....model_zoo.rcnn.rpn import RPN
label_desc: Use image node_id 4 for calling the DoubleHeadRCNN obj's underlying member method code with example usage: obj._pyramid_roi_feats(F, features, rpn_rois, roi_size, strides, roi_mode, roi_canonical_scale, sampling_ratio, eps) and returns: pooled_roi_feats