class_docstr: Optimized GroupNorm for NHWC layout with optional Swish/SiLU fusion.
  
  There are two version of CUDA kernels under the hood: one pass and two
  passes. This operator contains a simple heuristic to choose algorithm.
  
  Limitations:
  
  * Designed for 32 groups, also tested with 16 groups, some other number
    of groups can also work but not guaranteed;
  * Supported number of channels C are:
  
      128, 256, 320, 448, 512, 640, 768, 896, 960, 1024, 1280, 1344, 1536,
      1792, 1920, 2048, 2240, 2560, 2688, 3072, 3136, 3584, 4096.
  
    One pass algorithm supports only channels mentioned above. Two pass
    algorithm might automatically support some other channels as well.
  * N/H/W do not have lower (except >0) and upper bound limitations;
  
  All the unsupported cases will be forwarded to PyTorch implementation.
class_docstr_tok: ["Optimized","GroupNorm","for","NHWC","layout","with","optional","Swish\/SiLU","fusion",".","There","are","two","version","of","CUDA","kernels","under","the","hood",":","one","pass","and","two","passes",".","This","operator","contains","a","simple","heuristic","to","choose","algorithm",".","Limitations",":","*","Designed","for","32","groups",",","also","tested","with","16","groups",",","some","other","number","of","groups","can","also","work","but","not","guaranteed",";","*","Supported","number","of","channels","C","are",":","128",",","256",",","320",",","448",",","512",",","640",",","768",",","896",",","960",",","1024",",","1280",",","1344",",","1536",",","1792",",","1920",",","2048",",","2240",",","2560",",","2688",",","3072",",","3136",",","3584",",","4096",".","One","pass","algorithm","supports","only","channels","mentioned","above",".","Two","pass","algorithm","might","automatically","support","some","other","channels","as","well",".","*","N\/H\/W","do","not","have","lower","(","except",">","0",")","and","upper","bound","limitations",";","All","the","unsupported","cases","will","be","forwarded","to","PyTorch","implementation","."]
code: def __init__(
      self,
      num_groups: int,
      num_channels: int,
      eps: float = 1e-5,
      affine: bool = True,
      device=None,
      dtype=None,
      act="",
  ) -> None:
      factory_kwargs = {"device": device, "dtype": dtype}
      super().__init__()
      if num_channels % num_groups != 0:
          raise ValueError(
              "num_channels must be divisible by num_groups"
          )
  
      self.num_groups = num_groups
      self.num_channels = num_channels
      self.eps = eps
      self.affine = affine
      self.act = act.lower()
      if self.affine:
          self.weight = Parameter(
              torch.empty(num_channels, **factory_kwargs)
          )
          self.bias = Parameter(
              torch.empty(num_channels, **factory_kwargs)
          )
      else:
          self.register_parameter("weight", None)
          self.register_parameter("bias", None)
  
      self.reset_parameters()
      sm = torch.cuda.get_device_capability(device)
      self.sm = sm[0] * 10 + sm[1]
  
imports: import torch
  import torch.nn.functional
  import torch.nn.init
  import group_norm_cuda
  from torch import Tensor
  from torch.nn.parameter import Parameter
  from torch._dynamo import disable
  from functools import partial
label_desc: Use image node_id 1 to create a new GroupNorm object from inherited base classes: torch.nn with example: obj = GroupNorm(num_groups, num_channels, eps, affine, device, dtype, act)