title: Tokenizing Japanese text in R: Only first line of the specified column is tokenized
body: <p>I am trying to tokenize a collection of tweets with the Japanese tokenizer <code>RMeCab</code>, specifically the function <code>RMeCabDF</code> (for dataframes). </p>
  
  <p>The documentation states the following usage:</p>
  
  <blockquote>
    <p><strong>RMeCabDF</strong></p>
    
    <p><strong>Description</strong></p>
    
    <p>RMeCabDF takes data frames as the first argument, and analyzes the
    columns specified by the second argument. Blank data should be
    replaced with NA. If 1 is designated as the third argument, it returns
    each morpheme in its basic form.</p>
    
    <p><strong>Usage</strong></p>
    
    <p>RMeCabDF(dataf, coln, mypref, dic = "", mecabrc = "", etc = "")</p>
    
    <p><strong>Arguments</strong></p>
    
    <p><em>dataf</em>    data.frame </p>
    
    <p><em>coln</em>     Column number or name which include Japanese sentences</p>
    
    <p><em>mypref</em>   Default being 0, the same morphemic forms that appear on the text are returned. If 1 is designated, the basic forms of them are instead. </p>
    
    <p><em>dic</em>  to specify user dictionary, e.x. ishida.dic</p>
    
    <p><em>mecabrc</em>  not implemented (to specify mecab resource file) </p>
    
    <p><em>etc</em>  other options to mecab</p>
  </blockquote>
  
  <p>So following this, I use the following code to tokenize the column number <code>89</code> in the dataframe <code>trump_ja</code>:</p>
  
  <pre><code>trump_ja_tokens &lt;- RMeCabDF(trump_ja, coln = 89)
  </code></pre>
  
  <p>This results in a <code>List of 1</code> - but as you can see, the dataframe has 989 rows. </p>
  
  <p><a href="https://i.stack.imgur.com/5JM32.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/5JM32.png" alt="enter image description here"></a></p>
  
  <p>Where did my other rows go? </p>
  
  <p>Do I have to tokenize row by row? If so, is there any way to automate this process to avoid typing 1000 lines of code (or using Excel to produce 1000 lines of code)?</p>