docstr: Selective Kernel Convolution Module
  
  As described in Selective Kernel Networks (https://arxiv.org/abs/1903.06586) with some modifications.
  
  Largest change is the input split, which divides the input channels across each convolution path, this can
  be viewed as a grouping of sorts, but the output channel counts expand to the module level value. This keeps
  the parameter count from ballooning when the convolutions themselves don't have groups, but still provides
  a noteworthy increase in performance over similar param count models without this attention layer. -Ross W
  
  Args:
      in_channels (int):  module input (feature) channel count
      out_channels (int):  module output (feature) channel count
      kernel_size (int, list): kernel size for each convolution branch
      stride (int): stride for convolutions
      dilation (int): dilation for module as a whole, impacts dilation of each branch
      groups (int): number of groups for each branch
      rd_ratio (int, float): reduction factor for attention features
      keep_3x3 (bool): keep all branch convolution kernels as 3x3, changing larger kernels for dilations
      split_input (bool): split input channels evenly across each convolution branch, keeps param count lower,
          can be viewed as grouping by path, output expands to module out_channels count
      act_layer (nn.Module): activation layer to use
      norm_layer (nn.Module): batchnorm/norm layer to use
      aa_layer (nn.Module): anti-aliasing module
      drop_layer (nn.Module): spatial drop module in convs (drop block, etc)
docstr_tok: ["Selective","Kernel","Convolution","Module","As","described","in","Selective","Kernel","Networks","(","https",":","\/\/arxiv.org\/abs\/1903.06586",")","with","some","modifications",".","Largest","change","is","the","input","split",",","which","divides","the","input","channels","across","each","convolution","path",",","this","can","be","viewed","as","a","grouping","of","sorts",",","but","the","output","channel","counts","expand","to","the","module","level","value",".","This","keeps","the","parameter","count","from","ballooning","when","the","convolutions","themselves","do","n't","have","groups",",","but","still","provides","a","noteworthy","increase","in","performance","over","similar","param","count","models","without","this","attention","layer",".","-Ross","W","Args",":","in_channels","(","int",")",":","module","input","(","feature",")","channel","count","out_channels","(","int",")",":","module","output","(","feature",")","channel","count","kernel_size","(","int",",","list",")",":","kernel","size","for","each","convolution","branch","stride","(","int",")",":","stride","for","convolutions","dilation","(","int",")",":","dilation","for","module","as","a","whole",",","impacts","dilation","of","each","branch","groups","(","int",")",":","number","of","groups","for","each","branch","rd_ratio","(","int",",","float",")",":","reduction","factor","for","attention","features","keep_3x3","(","bool",")",":","keep","all","branch","convolution","kernels","as","3x3",",","changing","larger","kernels","for","dilations","split_input","(","bool",")",":","split","input","channels","evenly","across","each","convolution","branch",",","keeps","param","count","lower",",","can","be","viewed","as","grouping","by","path",",","output","expands","to","module","out_channels","count","act_layer","(","nn.Module",")",":","activation","layer","to","use","norm_layer","(","nn.Module",")",":","batchnorm\/norm","layer","to","use","aa_layer","(","nn.Module",")",":","anti-aliasing","module","drop_layer","(","nn.Module",")",":","spatial","drop","module","in","convs","(","drop","block",",","etc",")"]
code: def __init__(
      self,
      in_channels,
      out_channels=None,
      kernel_size=None,
      stride=1,
      dilation=1,
      groups=1,
      rd_ratio=1.0 / 16,
      rd_channels=None,
      rd_divisor=8,
      keep_3x3=True,
      split_input=True,
      act_layer=nn.ReLU,
      norm_layer=nn.BatchNorm2d,
      aa_layer=None,
      drop_layer=None,
  ):
      """Selective Kernel Convolution Module
  
      As described in Selective Kernel Networks (https://arxiv.org/abs/1903.06586) with some modifications.
  
      Largest change is the input split, which divides the input channels across each convolution path, this can
      be viewed as a grouping of sorts, but the output channel counts expand to the module level value. This keeps
      the parameter count from ballooning when the convolutions themselves don't have groups, but still provides
      a noteworthy increase in performance over similar param count models without this attention layer. -Ross W
  
      Args:
          in_channels (int):  module input (feature) channel count
          out_channels (int):  module output (feature) channel count
          kernel_size (int, list): kernel size for each convolution branch
          stride (int): stride for convolutions
          dilation (int): dilation for module as a whole, impacts dilation of each branch
          groups (int): number of groups for each branch
          rd_ratio (int, float): reduction factor for attention features
          keep_3x3 (bool): keep all branch convolution kernels as 3x3, changing larger kernels for dilations
          split_input (bool): split input channels evenly across each convolution branch, keeps param count lower,
              can be viewed as grouping by path, output expands to module out_channels count
          act_layer (nn.Module): activation layer to use
          norm_layer (nn.Module): batchnorm/norm layer to use
          aa_layer (nn.Module): anti-aliasing module
          drop_layer (nn.Module): spatial drop module in convs (drop block, etc)
      """
      super(SelectiveKernel, self).__init__()
      out_channels = out_channels or in_channels
      kernel_size = kernel_size or [
          3,
          5,
      ]  # default to one 3x3 and one 5x5 branch. 5x5 -> 3x3 + dilation
      _kernel_valid(kernel_size)
      if not isinstance(kernel_size, list):
          kernel_size = [kernel_size] * 2
      if keep_3x3:
          dilation = [dilation * (k - 1) // 2 for k in kernel_size]
          kernel_size = [3] * len(kernel_size)
      else:
          dilation = [dilation] * len(kernel_size)
      self.num_paths = len(kernel_size)
      self.in_channels = in_channels
      self.out_channels = out_channels
      self.split_input = split_input
      if self.split_input:
          assert in_channels % self.num_paths == 0
          in_channels = in_channels // self.num_paths
      groups = min(out_channels, groups)
  
      conv_kwargs = dict(
          stride=stride,
          groups=groups,
          act_layer=act_layer,
          norm_layer=norm_layer,
          aa_layer=aa_layer,
          drop_layer=drop_layer,
      )
      self.paths = nn.ModuleList(
          [
              ConvNormActAa(
                  in_channels,
                  out_channels,
                  kernel_size=k,
                  dilation=d,
                  **conv_kwargs,
              )
              for k, d in zip(kernel_size, dilation)
          ]
      )
  
      attn_channels = rd_channels or make_divisible(
          out_channels * rd_ratio, divisor=rd_divisor
      )
      self.attn = SelectiveKernelAttn(
          out_channels, self.num_paths, attn_channels
      )
  
file_path: pytorch-image-models/timm/layers/selective_kernel.py
imports: import torch
  from torch import nn
  from .conv_bn_act import ConvNormActAa
  from .helpers import make_divisible
  from .trace_utils import _assert
label_desc: Use image node_id 1 to create a new SelectiveKernel object from inherited base classes: nn with example: obj = SelectiveKernel(in_channels, out_channels, kernel_size, stride, dilation, groups, rd_ratio, rd_channels, rd_divisor, keep_3x3, split_input, act_layer, norm_layer, aa_layer, drop_layer)