The VAE-Transformer (Transformer) model, (Fig. ), integrates a VAE reparameterization into a full Transformer model as proposed in and implemented in . Transformers have taken the world of sequence2sequence learning by storm and have demonstrated their abilities to create extended relationships between sequence tokens that can be readily interpreted through attention head visualisations Brown et al.