class_docstr: Layer scale from [Touvron et al 2021] (https://arxiv.org/pdf/2103.17239.pdf).
  This rescales diagonally residual outputs close to 0 initially, then learnt.
class_docstr_tok: ["Layer","scale","from","[","Touvron","et","al","2021","]","(","https",":","\/\/arxiv.org\/pdf\/2103.17239.pdf",")",".","This","rescales","diagonally","residual","outputs","close","to","0","initially",",","then","learnt","."]
docstr: Args:
      channels (int): Size of  rescaling
      init (float, optional): Scale to default to (default: 0)
docstr_tok: ["Args",":","channels","(","int",")",":","Size","of","rescaling","init","(","float",",","optional",")",":","Scale","to","default","to","(","default",":","0",")"]
code: def __init__(self, channels: int, init: float = 0):
      r"""
      Args:
          channels (int): Size of  rescaling
          init (float, optional): Scale to default to (default: 0)
      """
      super().__init__()
      self.scale = nn.Parameter(
          torch.zeros(channels, requires_grad=True)
      )
      self.scale.data[:] = init
  
imports: import math
  import typing
  from typing import Any, Dict, List, Optional
  import torch
  from torch import nn
  from torch.nn import functional
label_desc: Use image node_id 1 to create a new _LayerScale object from inherited base classes: nn with example: obj = _LayerScale(channels, init)