In our continuous effort to contribute to the open-source community, features have been developed at Nuance and pushed to the Fairseq GitHub repository.  These try to overcome some of the challenges mentioned such as, facilitating copying of, especially rare or unseen, words from the input to summary, training speedups by improving Tensor Core utilization, and ensuring TorchScript compatibility of different Transformer configurations. Following, we will show an example of how to train a Transformer model with a Pointer Generator mechanism (Transformer-PG), which can copy words from the input.
How to build a Transformer model with a Pointer Generator mechanism
In this step-by-step guide, it is assumed the user has already installed PyTorch and Fairseq.
1. Create a vocabulary and extend it with source position markers:
These markers will allow the model to point to any word in the input sequence.
```python
vocab_size=
position_markers=512
export LC_ALL=C
cat train.src train.tgt |