title: read file from memory for regression (R)
body: <p>when trying to use the shglm function of the speedglm package I have a problem. As the file is too large to read into memory, I wanted to use a link function as outlined in the help pages for the package. The link function is </p>
  
  <pre><code>make.data&lt;-function(filename, chunksize,...){       
   conn&lt;-NULL
   function(reset=FALSE){
   if(reset){
     if(!is.null(conn)) close(conn)
     conn&lt;&lt;-file(filename,open="r")
   } else{
     rval&lt;-read.table(conn, nrows=chunksize,...)
     if ((nrow(rval)==0)) {
          close(conn)
          conn&lt;&lt;-NULL
          rval&lt;-NULL
     }
     return(rval)
   }
  </code></pre>
  
  <p>}
  }</p>
  
  <pre><code>load(ti.RData)
  </code></pre>
  
  <p>I then take my data fram (called ti) and write it to table</p>
  
  <pre><code>write.table(ti,"data1.txt",row.names=FALSE,col.names=FALSE)
  </code></pre>
  
  <p>as in the example here <a href="http://www.inside-r.org/packages/cran/speedglm/docs/shglm" rel="nofollow">http://www.inside-r.org/packages/cran/speedglm/docs/shglm</a>. Afterwards</p>
  
  <pre><code>da&lt;-make.data("data1.txt",chunksize=10000,col.names=colnames(ti))
  rm(ti)
  b1&lt;-shglm(T2D~factor(SIBCO)+factor(POCOD),datafun=da,family=binomial())
  </code></pre>
  
  <p>But I get an error </p>
  
  <pre><code>Error in dev.resids(y, mu, weights) : 
    argument mu must be a numeric vector of length 1 or length 802
  </code></pre>
  
  <p>I am happy to upload my data set but can somebody maybe roughly tell me where to start debugging? I think when reading in data1.txt through the link function ( with the read.table) some factors in the original data frame are by this operation converted to integers. This is the reason I put factor around the variables. Any suggestion wpould be very helpful</p>