We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
ssn_head.py device unaligned
**Describe the bug**

I trained a SSN successfully on a new dataset. But when I test the checkpoint, an error raised. And I think it's a inner bug.


**Reproduction**

1. What command or script did you run?

```
PYTHONPATH=$PWD:$PYTHONPATH mim test mmaction configs/localization/ssn/ssn_r50_450e_dfmad_rgb_test.py --checkpoint work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth --gpus 1 --eval mAP
```


**Environment**
```
sys.platform: linux
Python: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]
CUDA available: True
GPU 0,1: NVIDIA GeForce RTX 2080 Ti
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.5.r11.5/compiler.30672275_0
GCC: gcc (Ubuntu 8.4.0-3ubuntu2) 8.4.0
PyTorch: 1.10.1
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.2
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

TorchVision: 0.11.2
OpenCV: 4.5.4
MMCV: 1.4.1
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.3
MMAction2: 0.20.0+3758171
```

**Error traceback**
```shell
Testing command is python /home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py configs/localization/ssn/ssn_r50_450e_dfmad_rgb_test.py work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth --launcher none --eval mAP. 
2021-12-24 12:03:40,264 - mmaction - INFO - 13 out of 13 videos are valid.
2021-12-24 12:03:40,290 - mmaction - INFO - SSNDataset: proposal file my_data/dfmad70/proposals/dfmad_bmn_test_100_proposal_list.txt parsed.
load checkpoint from local path: work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth
[                                                  ] 0/13, elapsed: 0s, ETA:Traceback (most recent call last):
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py", line 365, in <module>
    main()
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py", line 350, in main
    outputs = inference_pytorch(args, cfg, distributed, data_loader)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py", line 161, in inference_pytorch
    outputs = single_gpu_test(model, data_loader)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmcv/engine/test.py", line 33, in single_gpu_test
    result = model(return_loss=False, **data)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmcv/parallel/data_parallel.py", line 50, in forward
    return super().forward(*inputs, **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/models/localizers/base.py", line 166, in forward
    return self.forward_test(*args, **kwargs)
  File "/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/models/localizers/ssn.py", line 120, in forward_test
    bbox_preds[:, :, 0] * reg_norm_consts[1, 0] +
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```

**Bug fix**
the bug come from here:
https://github.com/open-mmlab/mmaction2/blob/6f98109b682c5689dc0548a98b89ed1c3b956ddb/mmaction/models/localizers/ssn.py#L114-L121
We can see that all tensor are transformed to ``cpu().numpy()`` except the ``reg_norm_consts``.
Adding the below line solved the error:
```python
reg_norm_consts = reg_norm_consts.cpu().numpy()
```
I'd like to create a PR if need.

--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `mmaction/models/localizers/ssn.py`
Content:
```
1 # Copyright (c) OpenMMLab. All rights reserved.
2 import torch
3 import torch.nn as nn
4 
5 from .. import builder
6 from ..builder import LOCALIZERS
7 from .base import BaseTAGClassifier
8 
9 
10 @LOCALIZERS.register_module()
11 class SSN(BaseTAGClassifier):
12     """Temporal Action Detection with Structured Segment Networks.
13 
14     Args:
15         backbone (dict): Config for building backbone.
16         cls_head (dict): Config for building classification head.
17         in_channels (int): Number of channels for input data.
18             Default: 3.
19         spatial_type (str): Type of spatial pooling.
20             Default: 'avg'.
21         dropout_ratio (float): Ratio of dropout.
22             Default: 0.5.
23         loss_cls (dict): Config for building loss.
24             Default: ``dict(type='SSNLoss')``.
25         train_cfg (dict | None): Config for training. Default: None.
26         test_cfg (dict | None): Config for testing. Default: None.
27     """
28 
29     def __init__(self,
30                  backbone,
31                  cls_head,
32                  in_channels=3,
33                  spatial_type='avg',
34                  dropout_ratio=0.5,
35                  loss_cls=dict(type='SSNLoss'),
36                  train_cfg=None,
37                  test_cfg=None):
38 
39         super().__init__(backbone, cls_head, train_cfg, test_cfg)
40 
41         self.is_test_prepared = False
42         self.in_channels = in_channels
43 
44         self.spatial_type = spatial_type
45         if self.spatial_type == 'avg':
46             self.pool = nn.AvgPool2d((7, 7), stride=1, padding=0)
47         elif self.spatial_type == 'max':
48             self.pool = nn.MaxPool2d((7, 7), stride=1, padding=0)
49         else:
50             self.pool = None
51 
52         self.dropout_ratio = dropout_ratio
53         if self.dropout_ratio != 0:
54             self.dropout = nn.Dropout(p=self.dropout_ratio)
55         else:
56             self.dropout = None
57         self.loss_cls = builder.build_loss(loss_cls)
58 
59     def forward_train(self, imgs, proposal_scale_factor, proposal_type,
60                       proposal_labels, reg_targets, **kwargs):
61         """Define the computation performed at every call when training."""
62         imgs = imgs.reshape((-1, self.in_channels) + imgs.shape[4:])
63 
64         x = self.extract_feat(imgs)
65 
66         if self.pool:
67             x = self.pool(x)
68         if self.dropout is not None:
69             x = self.dropout(x)
70 
71         activity_scores, completeness_scores, bbox_preds = self.cls_head(
72             (x, proposal_scale_factor))
73 
74         loss = self.loss_cls(activity_scores, completeness_scores, bbox_preds,
75                              proposal_type, proposal_labels, reg_targets,
76                              self.train_cfg)
77         loss_dict = dict(**loss)
78 
79         return loss_dict
80 
81     def forward_test(self, imgs, relative_proposal_list, scale_factor_list,
82                      proposal_tick_list, reg_norm_consts, **kwargs):
83         """Define the computation performed at every call when testing."""
84         num_crops = imgs.shape[0]
85         imgs = imgs.reshape((num_crops, -1, self.in_channels) + imgs.shape[3:])
86         num_ticks = imgs.shape[1]
87 
88         output = []
89         minibatch_size = self.test_cfg.ssn.sampler.batch_size
90         for idx in range(0, num_ticks, minibatch_size):
91             chunk = imgs[:, idx:idx +
92                          minibatch_size, :, :, :].view((-1, ) + imgs.shape[2:])
93             x = self.extract_feat(chunk)
94             if self.pool:
95                 x = self.pool(x)
96             # Merge crop to save memory.
97             x = x.reshape((num_crops, x.size(0) // num_crops, -1)).mean(dim=0)
98             output.append(x)
99         output = torch.cat(output, dim=0)
100 
101         relative_proposal_list = relative_proposal_list.squeeze(0)
102         proposal_tick_list = proposal_tick_list.squeeze(0)
103         scale_factor_list = scale_factor_list.squeeze(0)
104         reg_norm_consts = reg_norm_consts.squeeze(0)
105 
106         if not self.is_test_prepared:
107             self.is_test_prepared = self.cls_head.prepare_test_fc(
108                 self.cls_head.consensus.num_multipliers)
109 
110         (output, activity_scores, completeness_scores,
111          bbox_preds) = self.cls_head(
112              (output, proposal_tick_list, scale_factor_list), test_mode=True)
113 
114         relative_proposal_list = relative_proposal_list.cpu().numpy()
115         activity_scores = activity_scores.cpu().numpy()
116         completeness_scores = completeness_scores.cpu().numpy()
117         if bbox_preds is not None:
118             bbox_preds = bbox_preds.view(-1, self.cls_head.num_classes, 2)
119             bbox_preds[:, :, 0] = (
120                 bbox_preds[:, :, 0] * reg_norm_consts[1, 0] +
121                 reg_norm_consts[0, 0])
122             bbox_preds[:, :, 1] = (
123                 bbox_preds[:, :, 1] * reg_norm_consts[1, 1] +
124                 reg_norm_consts[0, 1])
125             bbox_preds = bbox_preds.cpu().numpy()
126 
127         result = [
128             dict(
129                 relative_proposal_list=relative_proposal_list,
130                 activity_scores=activity_scores,
131                 completeness_scores=completeness_scores,
132                 bbox_preds=bbox_preds)
133         ]
134 
135         return result
136
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/mmaction/models/localizers/ssn.py b/mmaction/models/localizers/ssn.py
--- a/mmaction/models/localizers/ssn.py
+++ b/mmaction/models/localizers/ssn.py
@@ -114,6 +114,7 @@
         relative_proposal_list = relative_proposal_list.cpu().numpy()
         activity_scores = activity_scores.cpu().numpy()
         completeness_scores = completeness_scores.cpu().numpy()
+        reg_norm_consts = reg_norm_consts.cpu().numpy()
         if bbox_preds is not None:
             bbox_preds = bbox_preds.view(-1, self.cls_head.num_classes, 2)
             bbox_preds[:, :, 0] = (

{"golden_diff": "diff --git a/mmaction/models/localizers/ssn.py b/mmaction/models/localizers/ssn.py\n--- a/mmaction/models/localizers/ssn.py\n+++ b/mmaction/models/localizers/ssn.py\n@@ -114,6 +114,7 @@\n         relative_proposal_list = relative_proposal_list.cpu().numpy()\n         activity_scores = activity_scores.cpu().numpy()\n         completeness_scores = completeness_scores.cpu().numpy()\n+        reg_norm_consts = reg_norm_consts.cpu().numpy()\n         if bbox_preds is not None:\n             bbox_preds = bbox_preds.view(-1, self.cls_head.num_classes, 2)\n             bbox_preds[:, :, 0] = (\n", "issue": "ssn_head.py device unaligned\n**Describe the bug**\r\n\r\nI trained a SSN successfully on a new dataset. But when I test the checkpoint, an error raised. And I think it's a inner bug.\r\n\r\n\r\n**Reproduction**\r\n\r\n1. What command or script did you run?\r\n\r\n```\r\nPYTHONPATH=$PWD:$PYTHONPATH mim test mmaction configs/localization/ssn/ssn_r50_450e_dfmad_rgb_test.py --checkpoint work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth --gpus 1 --eval mAP\r\n```\r\n\r\n\r\n**Environment**\r\n```\r\nsys.platform: linux\r\nPython: 3.9.7 (default, Sep 16 2021, 13:09:58) [GCC 7.5.0]\r\nCUDA available: True\r\nGPU 0,1: NVIDIA GeForce RTX 2080 Ti\r\nCUDA_HOME: /usr/local/cuda\r\nNVCC: Build cuda_11.5.r11.5/compiler.30672275_0\r\nGCC: gcc (Ubuntu 8.4.0-3ubuntu2) 8.4.0\r\nPyTorch: 1.10.1\r\nPyTorch compiling details: PyTorch built with:\r\n  - GCC 7.3\r\n  - C++ Version: 201402\r\n  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications\r\n  - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)\r\n  - OpenMP 201511 (a.k.a. OpenMP 4.5)\r\n  - LAPACK is enabled (usually provided by MKL)\r\n  - NNPACK is enabled\r\n  - CPU capability usage: AVX2\r\n  - CUDA Runtime 11.3\r\n  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37\r\n  - CuDNN 8.2\r\n  - Magma 2.5.2\r\n  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.10.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, \r\n\r\nTorchVision: 0.11.2\r\nOpenCV: 4.5.4\r\nMMCV: 1.4.1\r\nMMCV Compiler: GCC 7.3\r\nMMCV CUDA Compiler: 11.3\r\nMMAction2: 0.20.0+3758171\r\n```\r\n\r\n**Error traceback**\r\n```shell\r\nTesting command is python /home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py configs/localization/ssn/ssn_r50_450e_dfmad_rgb_test.py work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth --launcher none --eval mAP. \r\n2021-12-24 12:03:40,264 - mmaction - INFO - 13 out of 13 videos are valid.\r\n2021-12-24 12:03:40,290 - mmaction - INFO - SSNDataset: proposal file my_data/dfmad70/proposals/dfmad_bmn_test_100_proposal_list.txt parsed.\r\nload checkpoint from local path: work_dirs/ssn_r50_450e_dfmad_rgb/latest.pth\r\n[                                                  ] 0/13, elapsed: 0s, ETA:Traceback (most recent call last):\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py\", line 365, in <module>\r\n    main()\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py\", line 350, in main\r\n    outputs = inference_pytorch(args, cfg, distributed, data_loader)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/.mim/tools/test.py\", line 161, in inference_pytorch\r\n    outputs = single_gpu_test(model, data_loader)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmcv/engine/test.py\", line 33, in single_gpu_test\r\n    result = model(return_loss=False, **data)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/modules/module.py\", line 1102, in _call_impl\r\n    return forward_call(*input, **kwargs)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmcv/parallel/data_parallel.py\", line 50, in forward\r\n    return super().forward(*inputs, **kwargs)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/parallel/data_parallel.py\", line 166, in forward\r\n    return self.module(*inputs[0], **kwargs[0])\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/torch/nn/modules/module.py\", line 1102, in _call_impl\r\n    return forward_call(*input, **kwargs)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/models/localizers/base.py\", line 166, in forward\r\n    return self.forward_test(*args, **kwargs)\r\n  File \"/home/louis/miniconda3/envs/open-mmlab/lib/python3.9/site-packages/mmaction/models/localizers/ssn.py\", line 120, in forward_test\r\n    bbox_preds[:, :, 0] * reg_norm_consts[1, 0] +\r\nRuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!\r\n```\r\n\r\n**Bug fix**\r\nthe bug come from here:\r\nhttps://github.com/open-mmlab/mmaction2/blob/6f98109b682c5689dc0548a98b89ed1c3b956ddb/mmaction/models/localizers/ssn.py#L114-L121\r\nWe can see that all tensor are transformed to ``cpu().numpy()`` except the ``reg_norm_consts``.\r\nAdding the below line solved the error:\r\n```python\r\nreg_norm_consts = reg_norm_consts.cpu().numpy()\r\n```\r\nI'd like to create a PR if need.\n", "before_files": [{"content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\nfrom .. import builder\nfrom ..builder import LOCALIZERS\nfrom .base import BaseTAGClassifier\n\n\n@LOCALIZERS.register_module()\nclass SSN(BaseTAGClassifier):\n    \"\"\"Temporal Action Detection with Structured Segment Networks.\n\n    Args:\n        backbone (dict): Config for building backbone.\n        cls_head (dict): Config for building classification head.\n        in_channels (int): Number of channels for input data.\n            Default: 3.\n        spatial_type (str): Type of spatial pooling.\n            Default: 'avg'.\n        dropout_ratio (float): Ratio of dropout.\n            Default: 0.5.\n        loss_cls (dict): Config for building loss.\n            Default: ``dict(type='SSNLoss')``.\n        train_cfg (dict | None): Config for training. Default: None.\n        test_cfg (dict | None): Config for testing. Default: None.\n    \"\"\"\n\n    def __init__(self,\n                 backbone,\n                 cls_head,\n                 in_channels=3,\n                 spatial_type='avg',\n                 dropout_ratio=0.5,\n                 loss_cls=dict(type='SSNLoss'),\n                 train_cfg=None,\n                 test_cfg=None):\n\n        super().__init__(backbone, cls_head, train_cfg, test_cfg)\n\n        self.is_test_prepared = False\n        self.in_channels = in_channels\n\n        self.spatial_type = spatial_type\n        if self.spatial_type == 'avg':\n            self.pool = nn.AvgPool2d((7, 7), stride=1, padding=0)\n        elif self.spatial_type == 'max':\n            self.pool = nn.MaxPool2d((7, 7), stride=1, padding=0)\n        else:\n            self.pool = None\n\n        self.dropout_ratio = dropout_ratio\n        if self.dropout_ratio != 0:\n            self.dropout = nn.Dropout(p=self.dropout_ratio)\n        else:\n            self.dropout = None\n        self.loss_cls = builder.build_loss(loss_cls)\n\n    def forward_train(self, imgs, proposal_scale_factor, proposal_type,\n                      proposal_labels, reg_targets, **kwargs):\n        \"\"\"Define the computation performed at every call when training.\"\"\"\n        imgs = imgs.reshape((-1, self.in_channels) + imgs.shape[4:])\n\n        x = self.extract_feat(imgs)\n\n        if self.pool:\n            x = self.pool(x)\n        if self.dropout is not None:\n            x = self.dropout(x)\n\n        activity_scores, completeness_scores, bbox_preds = self.cls_head(\n            (x, proposal_scale_factor))\n\n        loss = self.loss_cls(activity_scores, completeness_scores, bbox_preds,\n                             proposal_type, proposal_labels, reg_targets,\n                             self.train_cfg)\n        loss_dict = dict(**loss)\n\n        return loss_dict\n\n    def forward_test(self, imgs, relative_proposal_list, scale_factor_list,\n                     proposal_tick_list, reg_norm_consts, **kwargs):\n        \"\"\"Define the computation performed at every call when testing.\"\"\"\n        num_crops = imgs.shape[0]\n        imgs = imgs.reshape((num_crops, -1, self.in_channels) + imgs.shape[3:])\n        num_ticks = imgs.shape[1]\n\n        output = []\n        minibatch_size = self.test_cfg.ssn.sampler.batch_size\n        for idx in range(0, num_ticks, minibatch_size):\n            chunk = imgs[:, idx:idx +\n                         minibatch_size, :, :, :].view((-1, ) + imgs.shape[2:])\n            x = self.extract_feat(chunk)\n            if self.pool:\n                x = self.pool(x)\n            # Merge crop to save memory.\n            x = x.reshape((num_crops, x.size(0) // num_crops, -1)).mean(dim=0)\n            output.append(x)\n        output = torch.cat(output, dim=0)\n\n        relative_proposal_list = relative_proposal_list.squeeze(0)\n        proposal_tick_list = proposal_tick_list.squeeze(0)\n        scale_factor_list = scale_factor_list.squeeze(0)\n        reg_norm_consts = reg_norm_consts.squeeze(0)\n\n        if not self.is_test_prepared:\n            self.is_test_prepared = self.cls_head.prepare_test_fc(\n                self.cls_head.consensus.num_multipliers)\n\n        (output, activity_scores, completeness_scores,\n         bbox_preds) = self.cls_head(\n             (output, proposal_tick_list, scale_factor_list), test_mode=True)\n\n        relative_proposal_list = relative_proposal_list.cpu().numpy()\n        activity_scores = activity_scores.cpu().numpy()\n        completeness_scores = completeness_scores.cpu().numpy()\n        if bbox_preds is not None:\n            bbox_preds = bbox_preds.view(-1, self.cls_head.num_classes, 2)\n            bbox_preds[:, :, 0] = (\n                bbox_preds[:, :, 0] * reg_norm_consts[1, 0] +\n                reg_norm_consts[0, 0])\n            bbox_preds[:, :, 1] = (\n                bbox_preds[:, :, 1] * reg_norm_consts[1, 1] +\n                reg_norm_consts[0, 1])\n            bbox_preds = bbox_preds.cpu().numpy()\n\n        result = [\n            dict(\n                relative_proposal_list=relative_proposal_list,\n                activity_scores=activity_scores,\n                completeness_scores=completeness_scores,\n                bbox_preds=bbox_preds)\n        ]\n\n        return result\n", "path": "mmaction/models/localizers/ssn.py"}], "after_files": [{"content": "# Copyright (c) OpenMMLab. All rights reserved.\nimport torch\nimport torch.nn as nn\n\nfrom .. import builder\nfrom ..builder import LOCALIZERS\nfrom .base import BaseTAGClassifier\n\n\n@LOCALIZERS.register_module()\nclass SSN(BaseTAGClassifier):\n    \"\"\"Temporal Action Detection with Structured Segment Networks.\n\n    Args:\n        backbone (dict): Config for building backbone.\n        cls_head (dict): Config for building classification head.\n        in_channels (int): Number of channels for input data.\n            Default: 3.\n        spatial_type (str): Type of spatial pooling.\n            Default: 'avg'.\n        dropout_ratio (float): Ratio of dropout.\n            Default: 0.5.\n        loss_cls (dict): Config for building loss.\n            Default: ``dict(type='SSNLoss')``.\n        train_cfg (dict | None): Config for training. Default: None.\n        test_cfg (dict | None): Config for testing. Default: None.\n    \"\"\"\n\n    def __init__(self,\n                 backbone,\n                 cls_head,\n                 in_channels=3,\n                 spatial_type='avg',\n                 dropout_ratio=0.5,\n                 loss_cls=dict(type='SSNLoss'),\n                 train_cfg=None,\n                 test_cfg=None):\n\n        super().__init__(backbone, cls_head, train_cfg, test_cfg)\n\n        self.is_test_prepared = False\n        self.in_channels = in_channels\n\n        self.spatial_type = spatial_type\n        if self.spatial_type == 'avg':\n            self.pool = nn.AvgPool2d((7, 7), stride=1, padding=0)\n        elif self.spatial_type == 'max':\n            self.pool = nn.MaxPool2d((7, 7), stride=1, padding=0)\n        else:\n            self.pool = None\n\n        self.dropout_ratio = dropout_ratio\n        if self.dropout_ratio != 0:\n            self.dropout = nn.Dropout(p=self.dropout_ratio)\n        else:\n            self.dropout = None\n        self.loss_cls = builder.build_loss(loss_cls)\n\n    def forward_train(self, imgs, proposal_scale_factor, proposal_type,\n                      proposal_labels, reg_targets, **kwargs):\n        \"\"\"Define the computation performed at every call when training.\"\"\"\n        imgs = imgs.reshape((-1, self.in_channels) + imgs.shape[4:])\n\n        x = self.extract_feat(imgs)\n\n        if self.pool:\n            x = self.pool(x)\n        if self.dropout is not None:\n            x = self.dropout(x)\n\n        activity_scores, completeness_scores, bbox_preds = self.cls_head(\n            (x, proposal_scale_factor))\n\n        loss = self.loss_cls(activity_scores, completeness_scores, bbox_preds,\n                             proposal_type, proposal_labels, reg_targets,\n                             self.train_cfg)\n        loss_dict = dict(**loss)\n\n        return loss_dict\n\n    def forward_test(self, imgs, relative_proposal_list, scale_factor_list,\n                     proposal_tick_list, reg_norm_consts, **kwargs):\n        \"\"\"Define the computation performed at every call when testing.\"\"\"\n        num_crops = imgs.shape[0]\n        imgs = imgs.reshape((num_crops, -1, self.in_channels) + imgs.shape[3:])\n        num_ticks = imgs.shape[1]\n\n        output = []\n        minibatch_size = self.test_cfg.ssn.sampler.batch_size\n        for idx in range(0, num_ticks, minibatch_size):\n            chunk = imgs[:, idx:idx +\n                         minibatch_size, :, :, :].view((-1, ) + imgs.shape[2:])\n            x = self.extract_feat(chunk)\n            if self.pool:\n                x = self.pool(x)\n            # Merge crop to save memory.\n            x = x.reshape((num_crops, x.size(0) // num_crops, -1)).mean(dim=0)\n            output.append(x)\n        output = torch.cat(output, dim=0)\n\n        relative_proposal_list = relative_proposal_list.squeeze(0)\n        proposal_tick_list = proposal_tick_list.squeeze(0)\n        scale_factor_list = scale_factor_list.squeeze(0)\n        reg_norm_consts = reg_norm_consts.squeeze(0)\n\n        if not self.is_test_prepared:\n            self.is_test_prepared = self.cls_head.prepare_test_fc(\n                self.cls_head.consensus.num_multipliers)\n\n        (output, activity_scores, completeness_scores,\n         bbox_preds) = self.cls_head(\n             (output, proposal_tick_list, scale_factor_list), test_mode=True)\n\n        relative_proposal_list = relative_proposal_list.cpu().numpy()\n        activity_scores = activity_scores.cpu().numpy()\n        completeness_scores = completeness_scores.cpu().numpy()\n        reg_norm_consts = reg_norm_consts.cpu().numpy()\n        if bbox_preds is not None:\n            bbox_preds = bbox_preds.view(-1, self.cls_head.num_classes, 2)\n            bbox_preds[:, :, 0] = (\n                bbox_preds[:, :, 0] * reg_norm_consts[1, 0] +\n                reg_norm_consts[0, 0])\n            bbox_preds[:, :, 1] = (\n                bbox_preds[:, :, 1] * reg_norm_consts[1, 1] +\n                reg_norm_consts[0, 1])\n            bbox_preds = bbox_preds.cpu().numpy()\n\n        result = [\n            dict(\n                relative_proposal_list=relative_proposal_list,\n                activity_scores=activity_scores,\n                completeness_scores=completeness_scores,\n                bbox_preds=bbox_preds)\n        ]\n\n        return result\n", "path": "mmaction/models/localizers/ssn.py"}]}