title: Copying many large files from R to a new SQL db
body: <p>I have 6,000 files on a server, each 11M rows of a single column. As I understand it there are 3 steps: </p>
  
  <ol>
  <li>Connect to a SQL DB</li>
  <li>Read in the files</li>
  <li>Write them to the SQL DB</li>
  </ol>
  
  <p>So I think I've connected to the db:</p>
  
  <pre><code>sqlconnect &lt;- DBI::dbConnect(RSQLite::SQLite(), "~/folder/portal-database-output.sqlite")
  my_db_file &lt;- "~/folder/portal-database-output.sqlite"
  my_db &lt;- src_sqlite(my_db_file, create = TRUE)
  </code></pre>
  
  <p>And then I need to read in each file and then copy it to the SQL DB. I was thinking of creating a list of files (all .Rds in the folder) and then doing a map or loop that creates a df, copies to the DB, and then repeats x6000. How can I code this?</p>
  
  <p>Should I instead be trying to do a Bulk Insert? If so, how do I do so?</p>
  
  <p>Finally, I haven't decided if the result should be a single table 6000 wide or 6000 tables -- I don't have enough experience with SQL to determine what would be easier.</p>