title: Mathematica running out of memory
body: <p>I'm trying to run the following program, which calculates roots of polynomials of degree up to d with coefficients only +1 or -1, and then store it into files.</p>
  
  <pre><code>d = 20; n = 18000; 
  f[z_, i_] := Sum[(2 Mod[Floor[(i - 1)/2^k], 2] - 1) z^(d - k), {k, 0, d}];
  </code></pre>
  
  <p>Here f[z,i] gives a polynomial in z with plus or minus signs counting in binary. Say d=2, we would have</p>
  
  <p>f[z,1] = -z<sup>2</sup> - z - 1 <br />
  f[z,2] = -z<sup>2</sup> - z + 1 <br />
  f[z,3] = -z<sup>2</sup> + z - 1 <br />
  f[z,4] = -z<sup>2</sup> + z + 1 <br /></p>
  
  <pre><code>DistributeDefinitions[d, n, f]
  
  ParallelDo[ 
              Do[ 
                       root = N[Root[f[z, i], j]];
                       {a, b} = Round[n ({Re[root], Im[root]}/1.5 + 1)/2];
              {i, 1, 2^d}],
  {j, 1, d}]
  </code></pre>
  
  <p>I realise reading this probably isn't too enjoyable, but it's relatively short anyway. I would've tried to cut down to the relevant parts, but here I really have no clue what the trouble is. I'm calculating all roots of f[z,i], and then just round them to make them correspond to a point in a n by n grid, and save that data in various files.</p>
  
  <p>For some reason, the memory usage in Mathematica creeps up until it fills all the memory (6 GB on this machine); then the computation continues extremely slowly; why is this?</p>
  
  <p>I am not sure what is using up the memory here - my only guess was the stream of files used up memory, but that's not the case: I tried appending data to 2GB files and there was no noticeable memory usage for that. There seems to be absolutely no reason for Mathematica to be using large amounts of memory here.</p>
  
  <p>For small values of d (15 for example), the behaviour is the following: I have 4 kernels running. As they all run through the ParallelDo loop (each doing a value of j at a time), the memory use increases, until they all finish going through that loop once. Then the next times they go through that loop, the memory use does not increase at all. The calculation eventually finishes and everything is fine.</p>
  
  <p>Also, quite importantly, once the calculation stops, the memory use does not go back down.
  If I start another calculation, the following happens:</p>
  
  <p>-If the previous calculation stopped when memory use was still increasing, it continues to increase (it might take a while to start increasing again, basically to get to the same point in the computation).</p>
  
  <p>-If the previous calculation stopped when memory use was not increasing, it does not increase further.</p>
  
  <p><strong>Edit:</strong> The issue seems to come from the relative complexity of f - changing it into some easier polynomial seems to fix the issue. I thought the problem might be that Mathematica remembers f[z,i] for specific values of i, but setting f[z,i] :=. just after calculating a root of f[z,i] complains that the assignment did not exist in the first place, and the memory is still used.</p>
  
  <p>It's quite puzzling really, as f is the only remaining thing I can imagine taking up memory, but defining f in the inner Do loop and clearing it each time after a root is calculated does not solve the problem.</p>