Extracting small word parts (quadgrams) from text using R I would like to extracts small word parts (quadgrams) from text. Example:

hello world


results in:

_hel, hell, ello, llo_, lo_w ...


This my basic attempt thus far:

text &lt;- "hello world"

number_of_characters &lt;- nchar(text)
quad_gram_list &lt;- ""[-1]

for (i in 1:number_of_characters) {

    end &lt;- i + 3
    if (end &gt; number_of_characters) {
        end &lt;- number_of_characters
    }

    temp &lt;- substring(text, i, end)

    if (nchar(temp) == 4) {
        quad_gram_list &lt;- append(quad_gram_list, temp)
    }
}


Any improvement suggestions would be very much appreciated.
Your loop approach works but seems a bit convoluted. For instance you could rewrite the for loop this way:

for (i in 1:number_of_characters) {

  end &lt;- i + 3

  if (end &lt;= number_of_characters) {
    temp &lt;- substring(text, i, end)
    quad_gram_list &lt;- append(quad_gram_list, temp)
  }
}


But anyway it's generally better to avoid loops in R.

Also note that:


you should use character(0) instead of ""[-1]
: is to be avoided because of undesired behaviour on edge cases. It's better to use seq_len().


Here are two alternative solutions:

1) Using base R:

unlist(lapply(seq_len(nchar(text) - 3), function(i) substring(text, i, i + 3)))
# [1] "hell" "ello" "llo " "lo w" "o wo" " wor" "worl" "orld"


2) Using tokenizers:

library(tokenizers)

tokenize_character_shingles(
  text,
  n = 4,
  strip_non_alphanum = FALSE,
  simplify = TRUE
)
# [1] "hell" "ello" "llo " "lo w" "o wo" " wor" "worl" "orld"