class_docstr: Handle a linear transform, like torch.nn.Linear_, and a given activation, in a single kernel.
  The whole transform: is :math:`y = activation(xA^T + b)`.
  
  This is typically significantly faster than PyTorch while using fp16 and non-sigmoid activations,
  as of September 2021.
  
  .. _torch.nn.Linear: https://pytorch.org/docs/stable/generated/torch.nn.Linear.html
class_docstr_tok: ["Handle","a","linear","transform",",","like","torch.nn.Linear_",",","and","a","given","activation",",","in","a","single","kernel",".","The","whole","transform",":","is",":","math",":","`","y","=","activation","(","xA^T","+","b",")","`",".","This","is","typically","significantly","faster","than","PyTorch","while","using","fp16","and","non-sigmoid","activations",",","as","of","September","2021",".","..","_torch.nn.Linear",":","https",":","\/\/pytorch.org\/docs\/stable\/generated\/torch.nn.Linear.html"]
code: def __init__(
      self,
      in_features: int,
      out_features: int,
      bias: bool = False,
      activation: Optional[Activation] = None,
      **_,
  ):
      super().__init__()
      self.weight = nn.Parameter(
          torch.empty(out_features, in_features), requires_grad=True
      )
      self.bias = (
          nn.Parameter(torch.empty(out_features), requires_grad=True)
          if bias
          else None
      )
  
      self._activation_index = get_triton_activation_index(activation)
      self.reset_parameters()
  
file_path: xformers/xformers/triton/fused_linear_layer.py
imports: import math
  from typing import Any, Optional
  import torch
  import torch.nn
  from torch.cuda.amp import custom_bwd, custom_fwd
  from xformers.components.activations import Activation
  from xformers.triton.k_activations import get_triton_activation_index
  from xformers.triton.k_fused_matmul_bw import fused_matmul_backward
  from xformers.triton.k_fused_matmul_fw import fused_matmul
label_desc: Use image node_id 1 to create a new FusedLinear object from inherited base classes: nn with example: obj = FusedLinear(in_features, out_features, bias, activation)