class_docstr: Implements BERT version of Adam algorithm with weight decay fix.
  Params:
      lr: learning rate
      warmup: portion of t_total for the warmup, -1  means no warmup. Default: -1
      t_total: total number of training steps for the learning
          rate schedule, -1  means constant learning rate of 1. (no warmup regardless of warmup setting). Default: -1
      schedule: schedule to use for the warmup (see above).
          Can be `'warmup_linear'`, `'warmup_constant'`, `'warmup_cosine'`, `'none'`, `None` or a `_LRSchedule` object (see below).
          If `None` or `'none'`, learning rate is always kept constant.
          Default : `'warmup_linear'`
      b1: Adams b1. Default: 0.9
      b2: Adams b2. Default: 0.999
      e: Adams epsilon. Default: 1e-6
      weight_decay: Weight decay. Default: 0.01
      max_grad_norm: Maximum norm for the gradients (-1 means no clipping). Default: 1.0
code: def get_lr(self):
      lr = []
      for group in self.param_groups:
          for p in group["params"]:
              state = self.state[p]
              if len(state) == 0:
                  return [0]
              lr_scheduled = group["lr"]
              lr_scheduled *= group["schedule"].get_lr(state["step"])
              lr.append(lr_scheduled)
      return lr
  
imports: import math
  import torch
  from torch.optim import Optimizer
  from torch.optim.optimizer import required
  from torch.nn.utils import clip_grad_norm_
  import logging
  import abc
  import sys
label_desc: Use image node_id 2 for calling the BertAdam obj's underlying member method code with example usage: obj.get_lr() and returns: lr, list