export LC_ALL=C
cat train.src train.tgt |
  tr -s '[:space:]' '\n' |
  sort |
  uniq -c |
  sort -k1,1bnr -k2 |
  head -n "$((vocab_size - 4))" |
  awk '{ print $2 " " $1 }' > dict.pg.txt
python3 -c "[print(' 0'.format(n)) for n in range($position_markers)]" >> dict.pg.txt

This will create a file "dict.pg.txt" that contains the \<vocab_size> most frequent words followed by 512 position markers named from "\<unk-0>" to "\<unk-511>".

In case we have an input like

```python
src = "Hello, I'm The Dogtor"

it could happen that our model has been trained without the word "Dogtor" in its vocabulary. Therefore, when we feed this sequence into the model, it should be converted to:
src = "Hello, I'm The <unk-3>"

Now, "\" is part of our vocabulary and could be predicted by the model (this is where the pointer-generator comes in). In such a case, we will only need to post-process the output to replace "\" by the word at input position 3.