prompt: We are currently solving the following issue within our repository. Here is the issue text:
  --- BEGIN ISSUE ---
  iou3d failed when inference with gpu:1
  Thanks for your error report and we appreciate it a lot.
  
  **Checklist**
  1. I have searched related issues but cannot get the expected help.
  2. The bug has not been fixed in the latest version.
  
  **Describe the bug**
  Training on single GPU, when using default gpu (gpu:0) , everything is ok. 
  Switch to gpu:1, report `an illegal memory access was encountered mmdet3d/ops/iou3d/src/iou3d.cpp 121` during inference, however training is ok.
  
  **Reproduction**
  1. What command or script did you run?
  ```
  python tools/train.py CONFIG_PATH --gpu-ids 1
  ```
  2. Did you make any modifications on the code or config? Did you understand what you have modified?
  3. What dataset did you use?
  - kitti
  
  **Environment**
  
  1. Please run `python mmdet3d/utils/collect_env.py` to collect necessary environment infomation and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
      - How you installed PyTorch [e.g., pip, conda, source]
      - Other environment variables that may be related (such as `$PATH`, `$LD_LIBRARY_PATH`, `$PYTHONPATH`, etc.)
  
  **Error traceback**
  If applicable, paste the error trackback here.
  ```
  A placeholder for trackback.
  ```
  
  **Bug fix**
  If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
  
  
  --- END ISSUE ---
  
  Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
  --- BEGIN FILES ---
  Path: `mmdet3d/ops/iou3d/iou3d_utils.py`
  Content:
  ```
  1 import torch
  2 
  3 from . import iou3d_cuda
  4 
  5 
  6 def boxes_iou_bev(boxes_a, boxes_b):
  7     """
  8     :param boxes_a: (M, 5)
  9     :param boxes_b: (N, 5)
  10     :return:
  11         ans_iou: (M, N)
  12     """
  13 
  14     ans_iou = torch.cuda.FloatTensor(
  15         torch.Size((boxes_a.shape[0], boxes_b.shape[0]))).zero_()
  16 
  17     iou3d_cuda.boxes_iou_bev_gpu(boxes_a.contiguous(), boxes_b.contiguous(),
  18                                  ans_iou)
  19 
  20     return ans_iou
  21 
  22 
  23 def nms_gpu(boxes, scores, thresh):
  24     """
  25     :param boxes: (N, 5) [x1, y1, x2, y2, ry]
  26     :param scores: (N)
  27     :param thresh:
  28     :return:
  29     """
  30     # areas = (x2 - x1) * (y2 - y1)
  31     order = scores.sort(0, descending=True)[1]
  32 
  33     boxes = boxes[order].contiguous()
  34 
  35     keep = torch.LongTensor(boxes.size(0))
  36     num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh)
  37     return order[keep[:num_out].cuda()].contiguous()
  38 
  39 
  40 def nms_normal_gpu(boxes, scores, thresh):
  41     """
  42     :param boxes: (N, 5) [x1, y1, x2, y2, ry]
  43     :param scores: (N)
  44     :param thresh:
  45     :return:
  46     """
  47     # areas = (x2 - x1) * (y2 - y1)
  48     order = scores.sort(0, descending=True)[1]
  49 
  50     boxes = boxes[order].contiguous()
  51 
  52     keep = torch.LongTensor(boxes.size(0))
  53     num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh)
  54     return order[keep[:num_out].cuda()].contiguous()
  55
  ```
  --- END FILES ---
  
  Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.
  
  Here is an example:
  
  ```diff
  diff --git a/examples/server_async.py b/examples/server_async.py
  --- a/examples/server_async.py
  +++ b/examples/server_async.py
  @@ -313,4 +313,4 @@
   
   
   if __name__ == "__main__":
  -    asyncio.run(run_async_server("."), debug=True)
  +    asyncio.run(run_async_server(), debug=True)
  diff --git a/examples/server_sync.py b/examples/server_sync.py
  --- a/examples/server_sync.py
  +++ b/examples/server_sync.py
  @@ -313,5 +313,5 @@
   
   
   if __name__ == "__main__":
  -    server = run_sync_server(".")
  +    server = run_sync_server()
       server.shutdown()
  
  ```
  
golden_diff: diff --git a/mmdet3d/ops/iou3d/iou3d_utils.py b/mmdet3d/ops/iou3d/iou3d_utils.py
  --- a/mmdet3d/ops/iou3d/iou3d_utils.py
  +++ b/mmdet3d/ops/iou3d/iou3d_utils.py
  @@ -4,15 +4,17 @@
   
   
   def boxes_iou_bev(boxes_a, boxes_b):
  -    """
  -    :param boxes_a: (M, 5)
  -    :param boxes_b: (N, 5)
  -    :return:
  -        ans_iou: (M, N)
  -    """
  +    """Calculate boxes IoU in the bird view.
   
  -    ans_iou = torch.cuda.FloatTensor(
  -        torch.Size((boxes_a.shape[0], boxes_b.shape[0]))).zero_()
  +    Args:
  +        boxes_a (torch.Tensor): Input boxes a with shape (M, 5).
  +        boxes_b (torch.Tensor): Input boxes b with shape (N, 5).
  +
  +    Returns:
  +        ans_iou (torch.Tensor): IoU result with shape (M, N).
  +    """
  +    ans_iou = boxes_a.new_zeros(
  +        torch.Size((boxes_a.shape[0], boxes_b.shape[0])))
   
       iou3d_cuda.boxes_iou_bev_gpu(boxes_a.contiguous(), boxes_b.contiguous(),
                                    ans_iou)
  @@ -21,34 +23,41 @@
   
   
   def nms_gpu(boxes, scores, thresh):
  +    """Non maximum suppression on GPU.
  +
  +    Args:
  +        boxes (torch.Tensor): Input boxes with shape (N, 5).
  +        scores (torch.Tensor): Scores of predicted boxes with shape (N).
  +        thresh (torch.Tensor): Threshold of non maximum suppression.
  +
  +    Returns:
  +        torch.Tensor: Remaining indices with scores in descending order.
       """
  -    :param boxes: (N, 5) [x1, y1, x2, y2, ry]
  -    :param scores: (N)
  -    :param thresh:
  -    :return:
  -    """
  -    # areas = (x2 - x1) * (y2 - y1)
       order = scores.sort(0, descending=True)[1]
   
       boxes = boxes[order].contiguous()
   
  -    keep = torch.LongTensor(boxes.size(0))
  -    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh)
  -    return order[keep[:num_out].cuda()].contiguous()
  +    keep = boxes.new_zeros(boxes.size(0))
  +    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh, boxes.device.index)
  +    return order[keep[:num_out].cuda(boxes.device)].contiguous()
   
   
   def nms_normal_gpu(boxes, scores, thresh):
  +    """Normal non maximum suppression on GPU.
  +
  +    Args:
  +        boxes (torch.Tensor): Input boxes with shape (N, 5).
  +        scores (torch.Tensor): Scores of predicted boxes with shape (N).
  +        thresh (torch.Tensor): Threshold of non maximum suppression.
  +
  +    Returns:
  +        torch.Tensor: Remaining indices with scores in descending order.
       """
  -    :param boxes: (N, 5) [x1, y1, x2, y2, ry]
  -    :param scores: (N)
  -    :param thresh:
  -    :return:
  -    """
  -    # areas = (x2 - x1) * (y2 - y1)
       order = scores.sort(0, descending=True)[1]
   
       boxes = boxes[order].contiguous()
   
  -    keep = torch.LongTensor(boxes.size(0))
  -    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh)
  -    return order[keep[:num_out].cuda()].contiguous()
  +    keep = boxes.new_zeros(boxes.size(0))
  +    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh,
  +                                        boxes.device.index)
  +    return order[keep[:num_out].cuda(boxes.device)].contiguous()
  
verification_info: {"golden_diff": "diff --git a/mmdet3d/ops/iou3d/iou3d_utils.py b/mmdet3d/ops/iou3d/iou3d_utils.py\n--- a/mmdet3d/ops/iou3d/iou3d_utils.py\n+++ b/mmdet3d/ops/iou3d/iou3d_utils.py\n@@ -4,15 +4,17 @@\n \n \n def boxes_iou_bev(boxes_a, boxes_b):\n-    \"\"\"\n-    :param boxes_a: (M, 5)\n-    :param boxes_b: (N, 5)\n-    :return:\n-        ans_iou: (M, N)\n-    \"\"\"\n+    \"\"\"Calculate boxes IoU in the bird view.\n \n-    ans_iou = torch.cuda.FloatTensor(\n-        torch.Size((boxes_a.shape[0], boxes_b.shape[0]))).zero_()\n+    Args:\n+        boxes_a (torch.Tensor): Input boxes a with shape (M, 5).\n+        boxes_b (torch.Tensor): Input boxes b with shape (N, 5).\n+\n+    Returns:\n+        ans_iou (torch.Tensor): IoU result with shape (M, N).\n+    \"\"\"\n+    ans_iou = boxes_a.new_zeros(\n+        torch.Size((boxes_a.shape[0], boxes_b.shape[0])))\n \n     iou3d_cuda.boxes_iou_bev_gpu(boxes_a.contiguous(), boxes_b.contiguous(),\n                                  ans_iou)\n@@ -21,34 +23,41 @@\n \n \n def nms_gpu(boxes, scores, thresh):\n+    \"\"\"Non maximum suppression on GPU.\n+\n+    Args:\n+        boxes (torch.Tensor): Input boxes with shape (N, 5).\n+        scores (torch.Tensor): Scores of predicted boxes with shape (N).\n+        thresh (torch.Tensor): Threshold of non maximum suppression.\n+\n+    Returns:\n+        torch.Tensor: Remaining indices with scores in descending order.\n     \"\"\"\n-    :param boxes: (N, 5) [x1, y1, x2, y2, ry]\n-    :param scores: (N)\n-    :param thresh:\n-    :return:\n-    \"\"\"\n-    # areas = (x2 - x1) * (y2 - y1)\n     order = scores.sort(0, descending=True)[1]\n \n     boxes = boxes[order].contiguous()\n \n-    keep = torch.LongTensor(boxes.size(0))\n-    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh)\n-    return order[keep[:num_out].cuda()].contiguous()\n+    keep = boxes.new_zeros(boxes.size(0))\n+    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh, boxes.device.index)\n+    return order[keep[:num_out].cuda(boxes.device)].contiguous()\n \n \n def nms_normal_gpu(boxes, scores, thresh):\n+    \"\"\"Normal non maximum suppression on GPU.\n+\n+    Args:\n+        boxes (torch.Tensor): Input boxes with shape (N, 5).\n+        scores (torch.Tensor): Scores of predicted boxes with shape (N).\n+        thresh (torch.Tensor): Threshold of non maximum suppression.\n+\n+    Returns:\n+        torch.Tensor: Remaining indices with scores in descending order.\n     \"\"\"\n-    :param boxes: (N, 5) [x1, y1, x2, y2, ry]\n-    :param scores: (N)\n-    :param thresh:\n-    :return:\n-    \"\"\"\n-    # areas = (x2 - x1) * (y2 - y1)\n     order = scores.sort(0, descending=True)[1]\n \n     boxes = boxes[order].contiguous()\n \n-    keep = torch.LongTensor(boxes.size(0))\n-    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh)\n-    return order[keep[:num_out].cuda()].contiguous()\n+    keep = boxes.new_zeros(boxes.size(0))\n+    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh,\n+                                        boxes.device.index)\n+    return order[keep[:num_out].cuda(boxes.device)].contiguous()\n", "issue": "iou3d failed when inference with gpu:1\nThanks for your error report and we appreciate it a lot.\r\n\r\n**Checklist**\r\n1. I have searched related issues but cannot get the expected help.\r\n2. The bug has not been fixed in the latest version.\r\n\r\n**Describe the bug**\r\nTraining on single GPU, when using default gpu (gpu:0) , everything is ok. \r\nSwitch to gpu:1, report `an illegal memory access was encountered mmdet3d/ops/iou3d/src/iou3d.cpp 121` during inference, however training is ok.\r\n\r\n**Reproduction**\r\n1. What command or script did you run?\r\n```\r\npython tools/train.py CONFIG_PATH --gpu-ids 1\r\n```\r\n2. Did you make any modifications on the code or config? Did you understand what you have modified?\r\n3. What dataset did you use?\r\n- kitti\r\n\r\n**Environment**\r\n\r\n1. Please run `python mmdet3d/utils/collect_env.py` to collect necessary environment infomation and paste it here.\r\n2. You may add addition that may be helpful for locating the problem, such as\r\n    - How you installed PyTorch [e.g., pip, conda, source]\r\n    - Other environment variables that may be related (such as `$PATH`, `$LD_LIBRARY_PATH`, `$PYTHONPATH`, etc.)\r\n\r\n**Error traceback**\r\nIf applicable, paste the error trackback here.\r\n```\r\nA placeholder for trackback.\r\n```\r\n\r\n**Bug fix**\r\nIf you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!\r\n\n", "before_files": [{"content": "import torch\n\nfrom . import iou3d_cuda\n\n\ndef boxes_iou_bev(boxes_a, boxes_b):\n    \"\"\"\n    :param boxes_a: (M, 5)\n    :param boxes_b: (N, 5)\n    :return:\n        ans_iou: (M, N)\n    \"\"\"\n\n    ans_iou = torch.cuda.FloatTensor(\n        torch.Size((boxes_a.shape[0], boxes_b.shape[0]))).zero_()\n\n    iou3d_cuda.boxes_iou_bev_gpu(boxes_a.contiguous(), boxes_b.contiguous(),\n                                 ans_iou)\n\n    return ans_iou\n\n\ndef nms_gpu(boxes, scores, thresh):\n    \"\"\"\n    :param boxes: (N, 5) [x1, y1, x2, y2, ry]\n    :param scores: (N)\n    :param thresh:\n    :return:\n    \"\"\"\n    # areas = (x2 - x1) * (y2 - y1)\n    order = scores.sort(0, descending=True)[1]\n\n    boxes = boxes[order].contiguous()\n\n    keep = torch.LongTensor(boxes.size(0))\n    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh)\n    return order[keep[:num_out].cuda()].contiguous()\n\n\ndef nms_normal_gpu(boxes, scores, thresh):\n    \"\"\"\n    :param boxes: (N, 5) [x1, y1, x2, y2, ry]\n    :param scores: (N)\n    :param thresh:\n    :return:\n    \"\"\"\n    # areas = (x2 - x1) * (y2 - y1)\n    order = scores.sort(0, descending=True)[1]\n\n    boxes = boxes[order].contiguous()\n\n    keep = torch.LongTensor(boxes.size(0))\n    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh)\n    return order[keep[:num_out].cuda()].contiguous()\n", "path": "mmdet3d/ops/iou3d/iou3d_utils.py"}], "after_files": [{"content": "import torch\n\nfrom . import iou3d_cuda\n\n\ndef boxes_iou_bev(boxes_a, boxes_b):\n    \"\"\"Calculate boxes IoU in the bird view.\n\n    Args:\n        boxes_a (torch.Tensor): Input boxes a with shape (M, 5).\n        boxes_b (torch.Tensor): Input boxes b with shape (N, 5).\n\n    Returns:\n        ans_iou (torch.Tensor): IoU result with shape (M, N).\n    \"\"\"\n    ans_iou = boxes_a.new_zeros(\n        torch.Size((boxes_a.shape[0], boxes_b.shape[0])))\n\n    iou3d_cuda.boxes_iou_bev_gpu(boxes_a.contiguous(), boxes_b.contiguous(),\n                                 ans_iou)\n\n    return ans_iou\n\n\ndef nms_gpu(boxes, scores, thresh):\n    \"\"\"Non maximum suppression on GPU.\n\n    Args:\n        boxes (torch.Tensor): Input boxes with shape (N, 5).\n        scores (torch.Tensor): Scores of predicted boxes with shape (N).\n        thresh (torch.Tensor): Threshold of non maximum suppression.\n\n    Returns:\n        torch.Tensor: Remaining indices with scores in descending order.\n    \"\"\"\n    order = scores.sort(0, descending=True)[1]\n\n    boxes = boxes[order].contiguous()\n\n    keep = boxes.new_zeros(boxes.size(0))\n    num_out = iou3d_cuda.nms_gpu(boxes, keep, thresh, boxes.device.index)\n    return order[keep[:num_out].cuda(boxes.device)].contiguous()\n\n\ndef nms_normal_gpu(boxes, scores, thresh):\n    \"\"\"Normal non maximum suppression on GPU.\n\n    Args:\n        boxes (torch.Tensor): Input boxes with shape (N, 5).\n        scores (torch.Tensor): Scores of predicted boxes with shape (N).\n        thresh (torch.Tensor): Threshold of non maximum suppression.\n\n    Returns:\n        torch.Tensor: Remaining indices with scores in descending order.\n    \"\"\"\n    order = scores.sort(0, descending=True)[1]\n\n    boxes = boxes[order].contiguous()\n\n    keep = boxes.new_zeros(boxes.size(0))\n    num_out = iou3d_cuda.nms_normal_gpu(boxes, keep, thresh,\n                                        boxes.device.index)\n    return order[keep[:num_out].cuda(boxes.device)].contiguous()\n", "path": "mmdet3d/ops/iou3d/iou3d_utils.py"}]}