title: How to automate data importing, add time stamp, and standardize the number of rows in data files in R
body: <p>I have a folder that contains 1900 dataset (.dat), it's 1-day food-consumption data from 2012-2017. 
  They are named according to date like this:</p>
  
  <pre><code> FC-20120204.dat 
   FC-20120205.dat 
   FC-20120206.dat
  </code></pre>
  
  <p>In each dataset contains 1500 column (1500 variables), 1440 rows (1-minute data).</p>
  
  <p>My problems are these:</p>
  
  <ol>
  <li>A lot of the dataset is not complete i.e. only 899 rows, 1101 rows</li>
  <li>There is not date/time stamp, so it can be confusing</li>
  <li>I need only data from the time stamp and column 25th.</li>
  <li>These data is supposed to be continuous, so I have to eventually rbind them together.</li>
  </ol>
  
  <p>But first I figure I need to do:</p>
  
  <ol>
  <li>find a way to standardized all datasets to contain exactly 1440 rows.</li>
  </ol>
  
  <p>So far I've tried these from my previous question in stackoverflow:</p>
  
  <pre><code>files&lt;-list.files(pattern="FC")
  result &lt;- sapply(files, function(file) {
  temp &lt;- read.csv(file) # adjustments may be needed for headers, etc.
  temp[,25]
  })
  </code></pre>
  
  <p>it worked well until suddenly the R Studio crashed. I guess it's because I have 1900 dataset (the whole dataset is about 9 GB. My computer RAM is only 8 GB). So list.files function is not applicable, and i guess sapply and lapply is not applicable as well.</p>
  
  <p>I guess have to automate it one by one i.e. import one file at a time, add in the time stamp, standardize every file to contain 1440 rows, select the desired column, save it as a new .dat file, repeat with another file.</p>
  
  <p>After that, I can import all the 1900 new files (much smaller this time with only two columns) and rbind them together.</p>
  
  <p>Or is there an easier way to go about this?</p>
  
  <p>Update: Three of the dataset can be downloaded here: <a href="https://drive.google.com/drive/folders/17Rq1Vx21VqZhwYKWl9HLWZzxoZh2mxIh?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/drive/folders/17Rq1Vx21VqZhwYKWl9HLWZzxoZh2mxIh?usp=sharing</a>. The most important data is column C.</p>
  
  <p>or dropbox here <a href="https://www.dropbox.com/sh/q6c9w8kryn2by5z/AAD3rPnBgP5CyRy7E5eautJGa?dl=0" rel="nofollow noreferrer">https://www.dropbox.com/sh/q6c9w8kryn2by5z/AAD3rPnBgP5CyRy7E5eautJGa?dl=0</a></p>