class_docstr: ## BERT Embeddings
  
  For a given chunk of text $N$ this class generates BERT embeddings $        ext{B\small{ERT}}(N)$.
  $   ext{B\small{ERT}}(N)$ is the average of BERT embeddings of all the tokens in $N$.
docstr: In this implementation, we do not make chunks with a fixed number of tokens.
  One of the reasons is that this implementation uses character-level tokens and BERT
  uses its sub-word tokenizer.
  
  So this method will truncate the text to make sure there are no partial tokens.
  
  For instance, a chunk could be like `s a popular programming la`, with partial
  words (partial sub-word tokens) on the ends.
  We strip them off to get better BERT embeddings.
  As mentioned earlier this is not necessary if we broke chunks after tokenizing.
code: def _trim_chunk(chunk: str):
      """
      In this implementation, we do not make chunks with a fixed number of tokens.
      One of the reasons is that this implementation uses character-level tokens and BERT
      uses its sub-word tokenizer.
  
      So this method will truncate the text to make sure there are no partial tokens.
  
      For instance, a chunk could be like `s a popular programming la`, with partial
      words (partial sub-word tokens) on the ends.
      We strip them off to get better BERT embeddings.
      As mentioned earlier this is not necessary if we broke chunks after tokenizing.
      """
      # Strip whitespace
      stripped = chunk.strip()
      # Break words
      parts = stripped.split()
      # Remove first and last pieces
      stripped = stripped[len(parts[0]) : -len(parts[-1])]
  
      # Remove whitespace
      stripped = stripped.strip()
  
      # If empty return original string
      if not stripped:
          return chunk
      # Otherwise, return the stripped string
      else:
          return stripped
  
file_path: annotated_deep_learning_paper_implementations/labml_nn/transformers/retro/bert_embeddings.py
imports: from typing import List
  import torch
  from transformers import BertTokenizer, BertModel
  from labml import lab, monit
label_desc: Use image node_id 2 for calling the BERTChunkEmbeddings obj's underlying member method code with example usage: obj._trim_chunk(chunk) and returns: chunk, stripped