chunk_content: into_words=True because the text sequence has already been split into words. However, this doesn’t mean it is tokenized yet (even though it may look like it!), and you’ll need to further tokenize the words into subwords.
  
  
  	Copied
  tokenizer = AutoTokenizer.from_pretrained(model_checkpoint, add_prefix_space=True)
  You’ll also need to write a function to:
  Map each token to their respective word with the word_ids method.
  Ignore the special tokens b