class_docstr: Optimized GroupNorm for NHWC layout with optional Swish/SiLU fusion.
  
  There are two version of CUDA kernels under the hood: one pass and two
  passes. This operator contains a simple heuristic to choose algorithm.
  
  Limitations:
  
  * Designed for 32 groups, also tested with 16 groups, some other number
    of groups can also work but not guaranteed;
  * Supported number of channels C are:
  
      128, 256, 320, 448, 512, 640, 768, 896, 960, 1024, 1280, 1344, 1536,
      1792, 1920, 2048, 2240, 2560, 2688, 3072, 3136, 3584, 4096.
  
    One pass algorithm supports only channels mentioned above. Two pass
    algorithm might automatically support some other channels as well.
  * N/H/W do not have lower (except >0) and upper bound limitations;
  
  All the unsupported cases will be forwarded to PyTorch implementation.
class_docstr_tok: ["Optimized","GroupNorm","for","NHWC","layout","with","optional","Swish\/SiLU","fusion",".","There","are","two","version","of","CUDA","kernels","under","the","hood",":","one","pass","and","two","passes",".","This","operator","contains","a","simple","heuristic","to","choose","algorithm",".","Limitations",":","*","Designed","for","32","groups",",","also","tested","with","16","groups",",","some","other","number","of","groups","can","also","work","but","not","guaranteed",";","*","Supported","number","of","channels","C","are",":","128",",","256",",","320",",","448",",","512",",","640",",","768",",","896",",","960",",","1024",",","1280",",","1344",",","1536",",","1792",",","1920",",","2048",",","2240",",","2560",",","2688",",","3072",",","3136",",","3584",",","4096",".","One","pass","algorithm","supports","only","channels","mentioned","above",".","Two","pass","algorithm","might","automatically","support","some","other","channels","as","well",".","*","N\/H\/W","do","not","have","lower","(","except",">","0",")","and","upper","bound","limitations",";","All","the","unsupported","cases","will","be","forwarded","to","PyTorch","implementation","."]
code: def forward(self, input: Tensor) -> Tensor:
      can_use_nhwc_group_norm = self._check_legality(input)
  
      if can_use_nhwc_group_norm:
          channels = input.shape[1]
          hw = 1
          for i in range(2, len(input.shape)):
              hw *= input.shape[i]
          max_hw_one_pass = 1024 if self.sm >= 80 else 256
          if (
              hw >= 512 and channels in (3136, 3584, 4096)
          ) or hw > max_hw_one_pass:
              algo = group_norm_cuda.TwoPass
          else:
              algo = group_norm_cuda.OnePass
          return cuda_group_norm_nhwc(
              input,
              self.num_groups,
              self.weight,
              self.bias,
              self.eps,
              self.act,
              algo,
          )
      else:
          return torch_group_norm(
              input,
              self.num_groups,
              self.weight,
              self.bias,
              self.eps,
              self.act,
          )
  
imports: import torch
  import torch.nn.functional
  import torch.nn.init
  import group_norm_cuda
  from torch import Tensor
  from torch.nn.parameter import Parameter
  from torch._dynamo import disable
  from functools import partial
label_desc: Use image node_id 4 for calling the GroupNorm obj's underlying member method code with example usage: obj.forward(input) and returns: cuda_group_norm_nhwc, torch_group_norm