class_docstr: Samples the next tokens from the model's outputs.
  
  This layer does the following:
  1. Discard the hidden states that are not used for sampling (i.e., all
      tokens except the final one in each prompt).
  2. Compute the logits for the next tokens.
  3. Apply presence and frequency penalties.
  4. Apply temperature scaling.
  5. Apply top-p and top-k truncation.
  6. Sample the next tokens.
  Here, each sequence group within the batch can have different sampling
  parameters (e.g., sampling method, temperature, top-p, top-k, etc.).
class_docstr_tok: ["Samples","the","next","tokens","from","the","model","'s","outputs",".","This","layer","does","the","following",":","1",".","Discard","the","hidden","states","that","are","not","used","for","sampling","(","i.e.",",","all","tokens","except","the","final","one","in","each","prompt",")",".","2",".","Compute","the","logits","for","the","next","tokens",".","3",".","Apply","presence","and","frequency","penalties",".","4",".","Apply","temperature","scaling",".","5",".","Apply","top-p","and","top-k","truncation",".","6",".","Sample","the","next","tokens",".","Here",",","each","sequence","group","within","the","batch","can","have","different","sampling","parameters","(","e.g.",",","sampling","method",",","temperature",",","top-p",",","top-k",",","etc",".",")","."]
code: def forward(
      self,
      logits: torch.Tensor,
      input_metadata: InputMetadata,
      st_timestamp: Optional[float] = None,
  ) -> SamplerOutput:
      # Apply presence and frequency penalties.
      output_tokens = _get_output_tokens(input_metadata)
      presence_penalties, frequency_penalties = _get_penalties(
          input_metadata
      )
      logits = _apply_penalties(
          logits, output_tokens, presence_penalties, frequency_penalties
      )
      # Apply temperature scaling.
      temperatures = _get_temperatures(input_metadata)
      if any(t != 1.0 for t in temperatures):
          t = torch.tensor(
              temperatures, dtype=logits.dtype, device=logits.device
          )
          # Use in-place division to avoid creating a new tensor.
          logits.div_(t.unsqueeze(dim=1))
  
      # Apply top-p and top-k truncation.
      top_ps, top_ks = _get_top_p_top_k(input_metadata, self.vocab_size)
      do_top_p = any(p < 1.0 - _SAMPLING_EPS for p in top_ps)
      do_top_k = any(k != self.vocab_size for k in top_ks)
      if do_top_p or do_top_k:
          logits = _apply_top_p_top_k(logits, top_ps, top_ks)
  
      # We use float32 for probabilities and log probabilities.
      # Compute the probabilities.
      probs = torch.softmax(logits, dim=-1)
      # Compute the log probabilities.
      # Use log_softmax to ensure numerical stability.
      logprobs = torch.log_softmax(logits, dim=-1)
  
      # Sample the next tokens.
      return _sample(probs, logprobs, input_metadata, st_timestamp)
  
imports: from typing import Dict, List, Optional, Tuple
  import torch
  import torch.nn
  from bigdl.llm.vllm.model_executor.input_metadata import InputMetadata
  from bigdl.llm.vllm.sampling_params import SamplingParams, SamplingType
  from bigdl.llm.vllm.sequence import SamplerOutput, SequenceGroupMetadata, SequenceData, SequenceOutputs
  import time
label_desc: Use image node_id 2 for calling the BigDLSampler obj's underlying member method code with example usage: obj.forward(logits, input_metadata, st_timestamp) and returns: _sample