title: R: HUGE memory requirement applying variance structure to gls in nlme package
body: <p>I created a model using the <code>gls</code> function of the <code>nlme</code> package in R.
  I then attempted to add fixed variance structure to the model using the <code>weights</code> argument.</p>
  
  <p>However, I get an error about memory allocation that just seems, well, extreme.</p>
  
  <blockquote>
    <p>Error in glsEstimate(object, control = control) : 
      'Calloc' could not allocate memory (18446744073709551616 of 8 bytes)</p>
  </blockquote>
  
  <p><strong>Any suggestions about what to do with this??</strong></p>
  
  <hr>
  
  <p>Context:</p>
  
  <ul>
  <li><p>My Code: </p>
  
  <pre><code>mod &lt;- read.csv('mod.ht.dat.csv', head = T)
  dim(mod)
  [1] 90826     8
  
  library(nlme)
  lm3 &lt;- gls(HT ~ D * I(D^2), data = mod, na.action = na.omit, method = 'ML')
  vf1Fixed &lt;- varFixed(~D)
  lm2 &lt;- update(lm3, . ~ ., weights = vf1Fixed)
  Error in glsEstimate(object, control = control) : 
    'Calloc' could not allocate memory (18446744073709551616 of 8 bytes)
  </code></pre>
  
  <p>-- Note: model format is from <a href="http://link.springer.com/book/10.1007%2F978-0-387-87458-6" rel="nofollow noreferrer">Zuur et al. (2009)</a>.</p></li>
  <li><p>My memory usage (using code from <a href="https://stackoverflow.com/questions/1358003/tricks-to-manage-the-available-memory-in-an-r-session">here</a>) and memory limit: </p>
  
  <pre><code>&gt; lsos()
                   Type     Size    PrettySize  Rows Columns
  lm3               gls 12361512 [1] "11.8 Mb"    16      NA
  mod.ht.dat data.frame  4002768  [1] "3.8 Mb" 90826       8
  vf1Fixed     varFixed     1024    [1] "1 Kb"     0      NA
  
  &gt; memory.limit()
  [1] 8182
  </code></pre></li>
  <li><p>Session Info:</p>
  
  <pre><code>R version 3.3.1 (2016-06-21)
  Platform: x86_64-w64-mingw32/x64 (64-bit)
  Running under: Windows 7 x64 (build 7601) Service Pack 1
  </code></pre></li>
  </ul>
  
  <p>The necessary allocated memory seems ridiculously high for what I'm doing. </p>
  
  <p>I've modified the code by using <code>gls</code> directly (vs. <code>update</code>), I've placed <code>varFixed</code> in <em>and</em> outside of the model call itself, I've creatd a nw variable for D^2 before the model call, I've cleared my memory, I've restarted my computer, etc....Nothing seems to bring this HUGE number down.</p>
  
  <p><strong>Is it possible that adding this fixed variance structure to the model <em>really</em> is that memory intensive?? Or is perhaps something else going on here that I'm missing?...</strong></p>
  
  <hr>
  
  <p><strong>UPDATE</strong>:</p>
  
  <p>As requested in comments:</p>
  
  <pre><code>&gt;traceback()
  
  8: glsEstimate(object, control = control)
  7: Initialize.glsStruct(glsSt, dataMod, glsEstControl)
  6: Initialize(glsSt, dataMod, glsEstControl)
  5: gls(model = HT ~ D + I(D^2) + D:I(D^2), data = mod, method = "ML", 
     na.action = na.omit, weights = vf1Fixed)
  4: eval(expr, envir, enclos)
  3: eval(call, parent.frame())
  2: update.gls(lm3, . ~ ., weights = vf1Fixed)
  1: update(lm3, . ~ ., weights = vf1Fixed)
  
  &gt;dput(head(mod,5))
  
  structure(list(HT = c(3.7, 8.7, 10.1, 4, 8.7), SPEC = structure(c(53L, 
  53L, 53L, 53L, 53L), .Label = c("ACBA", "ACER", "ACRU", "AESY", 
  "AIAL", "ALJU", "AMAR", "BENI", "CACA", "CACO", "CACR", "CAFL", 
  "CAGL", "CAOL", "CAOV", "CAPA", "CARY", "CATO", "CECA", "CELA", 
  "CEOC", "CHVI", "COFL", "CRAT", "CRMA", "DIVI", "ELPU", "ELUM", 
  "EUAM", "FAGR", "FRAX", "GLTR", "HAVI", "ILAM", "ILDE", "ILOP", 
  "JUNI", "JUVI", "LIBE", "LIJA", "LISI", "LIST", "LITU", "LOMA", 
  "MAGR", "MATR", "MORU", "NYSY", "OSVI", "OXAR", "PATO", "PIEC", 
  "PITA", "PIVI", "PLOC", "PRSE", "QUAL", "QUCO", "QUER", "QUFA", 
  "QULY", "QUMA", "QUPH", "QURG", "QURU", "QUST", "QUVE", "RHCO", 
  "SAAL", "STGR", "ULAL", "ULAM", "ULRU", "UNKN", "VAAR", "VACC", 
  "VACO", "VAST", "VIAC", "VIBR", "VIPR", "VIRA", "VIRU"), class = "factor"), 
      D = c(4.1, 6.9, 7.4, 6.9, 13.7), plot = c(4L, 4L, 4L, 4L, 
      4L), tree_age = c(9L, 13L, 16L, 9L, 13L), Year = c(1933L, 
      1937L, 1940L, 1933L, 1937L), StaticLineID = c(1L, 1L, 1L, 
      2L, 2L), D2 = c(16.81, 47.61, 54.76, 47.61, 187.69)), .Names = c("HT", 
  "SPEC", "D", "plot", "tree_age", "Year", "StaticLineID", "D2"
  ), row.names = c(NA, 5L), class = "data.frame")
  </code></pre>
  
  <hr>
  
  <p><strong>Update 2:</strong></p>
  
  <p>Just to note: I tried applying a completely different type of variance structure to my data to see how my computer handled what I would have assumed to be a relatively similarly memory-intensive procedure.</p>
  
  <ul>
  <li><p>This time I added varIdent variance structure:</p>
  
  <pre><code>&gt;vf2 &lt;- varIdent(form = ~ 1 | SPEC)
  &gt;lm22 &lt;- update(lm3, . ~ ., weights = vf2)
  </code></pre></li>
  </ul>
  
  <p>Although it took forever to run (and ended up with a convergence error), it did not immediately produce the memory-allocation error as the former varFixed coding did.</p>