title: NaN loss during training of Faster R-CNN but not on SSD when using same config and data
body: <p>I have successfully trained a model with Tensorflow Object Detection API on my data, using pre-trained weights from <code>ssd_inception_v2_coco</code>.</p>
  
  <p>Now I am trying to train <code>faster_rcnn</code> model on the same data (the same TF Records, same label map and number of classes).</p>
  
  <p>Training runs for several steps with decreasing loss and then suddenly throws an error:
  <code>ERROR:tensorflow:Model diverged with loss = NaN.</code></p>
  
  <p>I have already tried:</p>
  
  <ul>
  <li>Decreasing learning rate (sometimes it increases the number of steps before error is thrown)</li>
  <li>Increasing batch size. Batch size = 24 is the largest that fits into memory. </li>
  <li>Checking inputs for NaNs or zero bounding boxes.</li>
  <li>Other <code>Faster R-CNN</code> pretrained models (<code>ResNet50, ResNet101, Inception V2</code>) from the model zoo.</li>
  </ul>
  
  <p>Any ideas what should I check / try next?</p>
  
  <p>Tensorflow version: 1.14.0</p>
  
  <p>The config file (adapted from <a href="https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_v2_coco.config" rel="nofollow noreferrer">https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/faster_rcnn_inception_v2_coco.config</a>)</p>
  
  <pre><code># Faster R-CNN with Inception v2, configuration for MSCOCO Dataset.
  # Users should configure the fine_tune_checkpoint field in the train config as
  # well as the label_map_path and input_path fields in the train_input_reader and
  # eval_input_reader. Search for "PATH_TO_BE_CONFIGURED" to find the fields that
  # should be configured.
  
  
  model {
    faster_rcnn {
      num_classes: 12
      image_resizer {
        fixed_shape_resizer {
          height: 300
          width: 300
        }
      }
      feature_extractor {
        type: 'faster_rcnn_inception_v2'
        first_stage_features_stride: 16
      }
      first_stage_anchor_generator {
        grid_anchor_generator {
          scales: [0.25, 0.5, 1.0, 2.0]
          aspect_ratios: [0.5, 1.0, 2.0]
          height_stride: 16
          width_stride: 16
        }
      }
      first_stage_box_predictor_conv_hyperparams {
        op: CONV
        regularizer {
          l2_regularizer {
            weight: 0.0
          }
        }
        initializer {
          truncated_normal_initializer {
            stddev: 0.01
          }
        }
      }
      first_stage_nms_score_threshold: 0.0
      first_stage_nms_iou_threshold: 0.7
      first_stage_max_proposals: 300
      first_stage_localization_loss_weight: 2.0
      first_stage_objectness_loss_weight: 1.0
      initial_crop_size: 14
      maxpool_kernel_size: 2
      maxpool_stride: 2
      second_stage_box_predictor {
        mask_rcnn_box_predictor {
          use_dropout: false
          dropout_keep_probability: 1.0
          fc_hyperparams {
            op: FC
            regularizer {
              l2_regularizer {
                weight: 0.0
              }
            }
            initializer {
              variance_scaling_initializer {
                factor: 1.0
                uniform: true
                mode: FAN_AVG
              }
            }
          }
        }
      }
      second_stage_post_processing {
        batch_non_max_suppression {
          score_threshold: 0.0
          iou_threshold: 0.6
          max_detections_per_class: 100
          max_total_detections: 300
        }
        score_converter: SOFTMAX
      }
      second_stage_localization_loss_weight: 2.0
      second_stage_classification_loss_weight: 1.0
    }
  }
  
  train_config: {
    batch_size: 20
    optimizer {
      momentum_optimizer: {
        learning_rate: {
          manual_step_learning_rate {
            initial_learning_rate: 0.00002
            schedule {
              step: 900000
              learning_rate: .00002
            }
            schedule {
              step: 1200000
              learning_rate: .000002
            }
          }
        }
        momentum_optimizer_value: 0.9
      }
      use_moving_average: false
    }
    gradient_clipping_by_norm: 10.0
    fine_tune_checkpoint: "/home/data/models/faster_rcnn_inception/model.ckpt"
    from_detection_checkpoint: true
    # Note: The below line limits the training process to 200K steps, which we
    # empirically found to be sufficient enough to train the COCO dataset. This
    # effectively bypasses the learning rate schedule (the learning rate will
    # never decay). Remove the below line to train indefinitely.
    num_steps: 200000
    data_augmentation_options {
      random_horizontal_flip {
      }
    }
  }
  
  train_input_reader: {
    tf_record_input_reader {
      input_path:  "/home/data/annotations/train.record"
    }
    label_map_path: "/home/data/annotations/label_map.pbtxt"
  }
  
  eval_config: {
    num_examples: 8000
    # Note: The below line limits the evaluation process to 10 evaluations.
    # Remove the below line to evaluate indefinitely.
    max_evals: 10
  }
  
  eval_input_reader: {
    tf_record_input_reader {
      input_path: "/home/data/annotations/val.record"
    }
    label_map_path: "/home/data/annotations/label_map.pbtxt"
    shuffle: false
    num_readers: 1
  }
  </code></pre>