title: Tokenizing issue
body: <p>I am trying to tokenize a sentence as follows.</p>
  
  <pre><code>Section &lt;- c("If an infusion reaction occurs, interrupt the infusion.")
  df &lt;- data.frame(Section)
  </code></pre>
  
  <p>When I tokenize using tidytext and the code below, </p>
  
  <pre><code>AA &lt;- df %&gt;%
    mutate(tokens = str_extract_all(df$Section, "([^\\s]+)"),
           locations = str_locate_all(df$Section, "([^\\s]+)"),
           locations = map(locations, as.data.frame)) %&gt;%
    select(-Section) %&gt;%
    unnest(tokens, locations) 
  </code></pre>
  
  <p>it gives me a result set as below (see image).</p>
  
  <p><a href="https://i.stack.imgur.com/NGtcl.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NGtcl.png" alt="R output for AA"></a></p>
  
  <p>How do i get the comma and the period as independent tokens as not part of 'occurs,' and 'infusion.' respectively, using tidytext. so my tokens should be </p>
  
  <pre><code>If
  an
  infusion
  reaction
  occurs
  ,
  interrupt
  the
  infusion
  .
  </code></pre>