class_docstr: This is just an example implementation with all
  operations explicited. This implementation is worse
  than pytorch, because pytorch is able to fuse some operations
  (eg the linear forward ...) that are decomposed here.
  
  The time measurements were made on the ViT-Giant setting:
  - A100/f16
  - input: [4440, 1536]
  - hidden: [4440, 4096]
class_docstr_tok: ["This","is","just","an","example","implementation","with","all","operations","explicited",".","This","implementation","is","worse","than","pytorch",",","because","pytorch","is","able","to","fuse","some","operations","(","eg","the","linear","forward","...",")","that","are","decomposed","here",".","The","time","measurements","were","made","on","the","ViT-Giant","setting",":","-","A100\/f16","-","input",":","[","4440",",","1536","]","-","hidden",":","[","4440",",","4096","]"]
code: def forward(cls, ctx, x, w1, b1, w2, b2, w3, b3):
      x1 = x @ w1.transpose(-2, -1) + b1  # 275us
      x2 = x @ w2.transpose(-2, -1) + b2  # 275us
      x3 = F.silu(x1)  # 62us
      x4 = x3 * x2  # 90us
      x5 = x4 @ w3.transpose(-2, -1) + b3  # 250us
  
      ctx.save_for_backward(
          x, w1, b1, w2, b2, w3, b3, x1, x2, x3, x4, x5
      )
      return x5
  
imports: from dataclasses import dataclass
  from typing import Dict, Optional, Sequence, Tuple, Union
  import torch
  import torch.nn.functional
  from torch import nn
  from .common import BaseOperator, get_xformers_operator, register_operator
  from .unbind import stack_or_none, unbind
label_desc: Use image node_id 2 for calling the _SwiGLUDecomposedFunc obj's underlying member method code with example usage: obj.forward(cls, ctx, x, w1, b1, w2, b2, w3, b3) and returns: x5