Creating consecutive words from a string
As @Michael Szyndel mentioned in a comment, Array#combination is the more appropriate method to use.

def get_separated_tokens query
  tokens = query.split
  (0..tokens.size).to_a.combination(2).map{ |i,j| tokens[i...j].join " " }
end