class_docstr:     ## Deterministic Parameter Free Project (DPFP)
  
      This is the new projection function $       extcolor{lightgreen}{\phi}$ introduced in the paper.
      DPFP projects $k$ of dimensionality $d_{key}$ to dimensionality $d_{dot} = 2 d_{key} 
  u$,
      where $
  u \in \{1, 2, ..., 2 d_{key} - 1 \}$ is a hyper-parameter.
  
      $$  extcolor{lightgreen}{\phi_{2 d_{key} (i - 1)  + j}(k)}
       =  ext{ReLU}\Big(ig[k, -kig]\Big)_{j}
                                  ext{ReLU}\Big(ig[k, -kig]\Big)_{i + j}$$
  
      where $ig[k, -kig]$ is the concatenation of $k$ and $-k$ to give a vector of
      size $2 d_{key}$, $i \in \{1, 2, ..., 
  u \}$, and $j \in \{1, 2, ..., 2 d_{key}\}$.
      $x_i$ is the $i$-th element of vector $x$ and is rolled around if
      $i$ is larger than the number of elements in $x$.
  
      Basically, it creates a new vector by multiplying elements of $[k, -k]$ shifted by $i$.
  
      This produces projections that are sparse (only a few elements of $phi$ are non-zero) and
      orthogonal ($       extcolor{lightgreen}{\phi(k^{(i)})} \cdot       extcolor{lightgreen}{\phi(k^{(j)})}
       pprox 0$ for most $i, j$
      unless $k^{(i)}$ and $k^{(j)}$ are very similar.
  
      ### Normalization
  
      Paper introduces a simple normalization for $       extcolor{lightgreen}{\phi}$,
  
      $$  extcolor{lightgreen}{\phi '(k)} =
       rac{      extcolor{lightgreen}{\phi(k)}}{\sum^{d_{dot}}_{j=1}     extcolor{lightgreen}{\phi(k)_j}}$$
  
      *Check the paper for derivation.*
      
code: def forward(self, k: torch.Tensor):
      # Get $\textcolor{lightgreen}{\phi(k)}$
      k = self.dpfp(k)
      # Normalize by $\sum^{d_{dot}}_{j=1} \textcolor{lightgreen}{\phi(k)_j}$
      return k / (torch.sum(k, dim=-1, keepdim=True) + self.eps)
  
imports: import torch
  from torch import nn
  from labml_helpers.module import Module
  from labml_nn.transformers.feed_forward import FeedForward
  from labml_nn.transformers.mha import PrepareForMultiHeadAttention
  from labml_nn.utils import clone_module_list
label_desc: Use image node_id 2 for calling the DPFP obj's underlying member method code with example usage: obj.forward(k) and returns: unknown