class_docstr: This is a multi-layer regression model that outputs UCB score.
  There are two modules in this model: MLP module and LinUCB module.
  The MLP module consists of bottom layers whic are trainable by torch optimizer().
  The LinUCB module is the last layer and it is not updated by optimizer but by matrix computations.
  MLP module Refer to paper https://arxiv.org/pdf/2012.01780.pdf.
  LinUCB module refer to paper https://arxiv.org/pdf/2012.01780.pdf.
  
  The reason to use matrix computations to update model parameters is to output uncertainty besides prediction.
  Example :
      Features(dim=9) --> deep_represent_layers --> Features(dim=3) --> LinUCB --> ucb score
  
      DeepRepresentLinUCBTrainer(
      (scorer): DeepRepresentLinearRegressionUCB(
          (deep_represent_layers): FullyConnectedNetwork(
          (dnn): Sequential(
              (0): Linear(in_features=9, out_features=6, bias=True)
              (1): ReLU()
              (2): Linear(in_features=6, out_features=3, bias=True)
              (3): Identity()
          )
          )
      )
      (loss_fn): MSELoss()
      )
  
  In this implementation,
  - ucb = pred_label + uncertainty
  - uncertainty = ucb_alpha * pred_sigma
  
  - pred_label is the predicted label.
  - pred_sigma reflects the variance associated with the predicted label.
  - ucb_alpha controls the balance between exploration and exploitation,
      - If ucb_alpha is not 0, uncertainty(ucb_alpha * pred_sigma) will be included in the final output.
      - If ucb_alpha is 0, uncertainty(ucb_alpha * pred_sigma) will not be included. The model only outputs a predicted label like a classical supervised model.
  - mlp_out_with_ones is the output from the deep_represent module (with extra column of ones appended for the bias term), also it is the input to the LinUCB module.
  - coefs serve as the top layer LinUCB module in this implementation.
      - it is crutial that coefs will not be updated by gradient back propagation.
      - coefs is defined by @property decorator in LinearRegressionUCB.
  
  Note in the current implementation the LinUCB coefficients are automatically re-computed at every training step.
  This can be costly for high-dimension LinUCB input (which is the output of `deep_represent_layers`),
  Thus, it's recommended to keep sizes[-1]) low.
code: def input_prototype(self) -> torch.Tensor:
      return torch.randn(1, self.raw_input_dim)
  
file_path: ReAgent/reagent/models/deep_represent_linucb.py
imports: import logging
  from typing import Dict, List, Optional
  import torch
  from reagent.models.fully_connected_network import ACTIVATION_MAP, FullyConnectedNetwork
  from reagent.models.linear_regression import batch_quadratic_form, LinearRegressionUCB
  from torch import nn
label_desc: Use image node_id 2 for calling the DeepRepresentLinearRegressionUCB obj's underlying member method code with example usage: obj.input_prototype() and returns: torch