title: r -- apply method for character counting for a frequency-weighted wordlist
body: <p>Ok, I have a list of words with their frequencies. There are many, many thousands of these. Here's a mini example:</p>
  <pre><code>w = c(&quot;abandon&quot;, &quot;break&quot;, &quot;fuzz&quot;, &quot;when&quot;)
  f = c(2, 10, 8, 200)
  df = data.frame(cbind(w, f))
  df
  
          w   f
  1 abandon   2
  2   break  10
  3    fuzz   8
  4    when 200
  </code></pre>
  <p>What I want to do is count the characters in each word and then aggregate the results. The <code>count_chars</code> function from the <code>dw4psy</code> package can do this for a given vector of strings. I've done this successfully by just creating a giant vector of strings from the word list (which has 10s of 1000s of words), as follows:</p>
  <pre><code>library(ds4psy) # for count_chars function 
  library(dplyr)
  
  w = c(&quot;abandon&quot;, &quot;break&quot;, &quot;fuzz&quot;, &quot;when&quot;)
  f = c(2, 10, 8, 200)
  df = data.frame(cbind(w, f))
  df$w = as.character(df$w)
  df$f = as.integer(df$f)
  
  # repword will repeat wrd frq times with no spaces between
  repword &lt;- function(frq, wrd) paste(rep(times=frq, x=wrd), collapse=&quot;&quot;)
  
  # now we create one giant vector of strings to do the counts on 
  # CAUTION -- uses lots of memory when you have 10s of 1000s of words
  mytext = paste(mapply(repword,  df$f, df$w))
  
  # get a table of letter counts
  mycounts = count_chars(mytext)
  
  # convert to data frame sorted by character
  mycounts.df &lt;- mycounts[order(names(mycounts))] %&gt;%
    as.data.frame()
  
  # sort by Freq in descending order
  mycounts.df %&gt;% 
    arrange(desc(Freq))
  </code></pre>
  <p>However, a colleague does not have enough memory for this brute force solution. So I tried to figure out how to do this word-by-word using foreach or mapply, but I am really stuck.</p>
  <p>One issue is that you need a vector that has every letter in it to combine them (so far as I can tell). So I create a dummy word with all letters in it, and then do some tweaks to keep it from counting the repeated letters each time.</p>
  <pre><code># create a dummy string that is a-z
  dummy = paste0(letters, collapse=&quot;&quot;)
  # now we create a count - it will be all 1s; we will subtract it every time
  dummycount = count_chars(dummy)
  
  
  countword &lt;- function(frq, wrd) {
    myword = paste0(dummy, wrd, collapse=&quot;&quot;)
    # subtract 1 from each letter to correct for dummy
    mycount = count_chars(myword) - dummycount 
    mycount = mycount * frq # multiply by frequency
    return(mycount)
  }
  
  totalcount = dummycount - 1 # set a table to zeroes
  
  
  foreach(frq = df$f, wrd = df$w) %do% {
   totalcount = totalcount + countword(frq, wrd)
  }
  </code></pre>
  <p>But this just doesn't work ... I get a weird result:</p>
  <pre><code>
  &gt; totalcount
  chars
   a  b  c  d  e  f  g  h  i  j  k  l  m  n  o  p  q  r  s  t  u  v  w  x  y  z 
  16 12 10  6  3  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0 
  </code></pre>
  <p>I would be very grateful for any advice!</p>