title: ideas to avoid hitting memory limit when using dbWriteTable to save an R data table inside a SQLite database
body: <p>a data frame that's small enough to be loaded into R still occasionally hits the memory-limit ceiling during a <code>dbWriteTable</code> call if it was near the maximum amount of RAM available.  i'm wondering if there's any better solution than reading the table into RAM in chunks like the code below?</p>
  
  <p>i'm trying to write code that will work on older computers, so i'm using the windows 32-bit version of R to re-create these memory errors..</p>
  
  <pre><code># this example will only work on a computer with at least 3GB of RAM
  # because it intentionally maxes out the 32-bit limit
  
  # create a data frame that's barely fits inside 32-bit R's memory capacity
  x &lt;- mtcars[ rep( seq( nrow( mtcars ) ) , 400000 ) , ]
  
  # check how many records this table contains..
  nrow( x )
  
  # create a connection to a SQLite database
  # not stored in memory
  library( RSQLite )
  tf &lt;- tempfile()
  db &lt;- dbConnect( SQLite() , tf )
  
  
  # storing `x` in the database with dbWriteTable breaks.
  # this line causes a memory error
  # dbWriteTable( db , 'x' , x )
  
  # but storing it in chunks works!
  chunks &lt;- 100
  
  starts.stops &lt;- floor( seq( 1 , nrow( x ) , length.out = chunks ) )
  
  
  for ( i in 2:( length( starts.stops ) )  ){
  
      if ( i == 2 ){
          rows.to.add &lt;- ( starts.stops[ i - 1 ] ):( starts.stops[ i ] )
      } else {
          rows.to.add &lt;- ( starts.stops[ i - 1 ] + 1 ):( starts.stops[ i ] )
      }
  
      # storing `x` in the database with dbWriteTable in chunks works.
      dbWriteTable( db , 'x' , x[ rows.to.add , ] , append = TRUE )
  }
  
  
  # and it's the correct number of lines.
  dbGetQuery( db , "select count(*) from x" )
  </code></pre>