title: data.table: preallocating memory for future columns
body: <p>We have a very large <code>data.table</code>, to which we append columns, mainly by <code>data.table.merge</code>. Occasionally, this triggers a <code>"Cannot allocate vector of size xx Gb"</code> error, even though we know that there is this amount of memory available on the system.</p>
  
  <p>Our suspicion is that this is due to the fact that this memory isn't part of a contiguous block, so we would like to somehow preallocate a larger chunk of RAM when creating the data.table. </p>
  
  <p>One obvious suggestion is to just create all the columns that will be eventually merged into our data.table from another one at the outset. However, this isn't necessarily going to work, because <code>merge</code> is designed not to overwrite the columns of the <code>DT1</code> with those of <code>DT2</code> having the same name, but to rename them such that both can be kept.</p>
  
  <p>Is there anything else that can be done?</p>
  
  <p>Minimal example:</p>
  
  <pre><code>x = data.table(a = 1:10, b=2:11)
  y = data.table(a = 1:10, c=2:11)
  
  # want this to happen in the most memory-efficient way possible 
  # and ideally without allocating new memory at all 
  # (i.e., want to be able to pre-allocate enough memory in x 
  # in line 1 to be able to do this)
  x = merge(x, y, by=a)
  </code></pre>