title: How to map token indices from the SQuAD data to tokens from BERT tokenizer?
body: <p>I am using the SQuaD dataset for answer span selection. After using the <code>BertTokenizer</code> to tokenize the passages, for some samples, the start and end indices of the answer don't match the real answer span position in the passage tokens anymore. How to solve this problem? One way is to modify the answer indices (also the training targets) accordingly? But how to do it?</p>