title: Why is allow.cartesian required at times when when joining data.tables with duplicate keys?
body: <p>I am trying to understand the logic of J() lookup when there're duplicate keys in a data.table in R.</p>
  
  <p>Here's a little experiment I have tried:</p>
  
  <pre><code>library(data.table)
  options(stringsAsFactors = FALSE)
  
  x &lt;- data.table(keyVar = c("a", "b", "c", "c"),
              value  = c(  1,   2,   3,   4))
  setkey(x, keyVar)
  
  y1 &lt;- data.frame(name = c("d", "c", "a"))
  x[J(y1$name), ]
  ## OK
  
  y2 &lt;- data.frame(name = c("d", "c", "a", "b"))
  x[J(y2$name), ]
  ## Error: see below
  
  x2 &lt;- data.table(keyVar = c("a", "b", "c"),
                   value  = c(  1,   2,   3))
  setkey(x2, keyVar)
  x2[J(y2$name), ]
  ## OK
  </code></pre>
  
  <p>The error message I am getting is :</p>
  
  <pre><code>Error in vecseq(f__, len__, if (allow.cartesian) NULL else as.integer(max(nrow(x),  :
  Join results in 5 rows; more than 4 = max(nrow(x),nrow(i)). Check for duplicate key
  values in i, each of which join to the same group in x over and over again. If that's
  ok, try including `j` and dropping `by` (by-without-by) so that j runs for each group
  to avoid the large allocation. If you are sure you wish to proceed, rerun with 
  allow.cartesian=TRUE. Otherwise, please search for this error message in the FAQ, Wiki, 
  Stack Overflow and datatable-help for advice.
  </code></pre>
  
  <p>I don't really understand this. I know I should avoid duplicate keys in a lookup function, I just want to gain some insight so I won't make any error in the future.</p>
  
  <p>Thanks a ton for help. This is a great tool.</p>