title: XML generation very slow and using lots of memory in Rails 4
body: <p>I'm generating an XML file to share data with another system. From my troubleshooting, I've found that this process is both slow and consuming lots of memory (getting lots of R14's on Heroku.)</p>
  
  <p>My index method on my Jobs Controller looks like this:</p>
  
  <pre><code>def index
    respond_to do |format|
      format.xml {@jobs = @user.jobs.includes(job_types: [:job_lines, :job_photos])}
      format.json 
      { 
        # More code here, this part is not the problem.
      }
    end
  end
  </code></pre>
  
  <p>My view (index.xml.builder) looks like this (I've removed a bunch of fields to keep the example smaller):</p>
  
  <pre><code>xml.instruct!
  xml.jobs do
    @jobs.each do |j|
      xml.job do
        xml.id             j.id
        xml.job_number     j.job_number
        xml.registration   j.registration
        xml.name           j.name
        xml.job_types do
          j.job_types.each do |t|
            xml.job_type do
              xml.id          t.id
              xml.job_id      t.job_id
              xml.type_number t.type_number
              xml.description t.description
              xml.job_lines do
                t.job_lines.each do |l|
                  xml.job_line do
                    xml.id          l.id
                    xml.line_number l.line_number
                    xml.job_type_id l.job_type_id
                    xml.line_type   l.line_type
                    xml.type_number l.type_number
                    xml.description l.description
                    xml.part_number l.part_number
                  end # job_line node
                end # job_lines.each
              end # job_lines node
              xml.job_photos do
                t.job_photos.each do |p|
                  xml.job_photo do
                    xml.id          p.id
                    xml.pcid        p.pcid
                    xml.job_type_id p.job_type_id
                    xml.image_url   p.image.url
                  end # job_line node
                end # job_lines.each
              end # job_lines node
            end # job_type
          end # job_types.each
        end # job_types node
      end # job node
    end # @jobs.each
  end # jobs node
  </code></pre>
  
  <p>The generated XML file is not small (it's about 100kB). Running on Heroku, their Scout tool tells me that this process is often taking 4-6 seconds to run. Also, despite only running 1 worker, with 4 threads (in Puma) this part of my code is consuming all my memory. In scout, I can see that it's "Max Allocations" are as high as 10M compared with my next worst method which is only 500k allocations.</p>
  
  <p>Can anyone tell me what I'm doing wrong? Is there a more efficient (in terms of speed and memory usage) way for me to generate XML?</p>
  
  <p>Any help would be appreciated.</p>
  
  <p><strong>EDIT 1</strong></p>
  
  <p>I've tried building the XML manually like this:</p>
  
  <pre><code>joblist.each do |j|
    result &lt;&lt; "  &lt;job&gt;\n"
    result &lt;&lt; "    &lt;id&gt;" &lt;&lt; j.id.to_s &lt;&lt; "&lt;/id&gt;\n"
    result &lt;&lt; "    &lt;job_number&gt;"    &lt;&lt; j.job_number.to_s &lt;&lt; "&lt;/job_number&gt;\n"
    # Lots more lines removed
  end 
  </code></pre>
  
  <p>This has given me some improvements. My largest allocations is now 1.8M. I'm close to Heroku's limit (reached a max of 500MB out of the 512MB limit over 24 hours).</p>
  
  <p>I am still only running 1 Worker with 4 threads.If I can I'd like to get the memory down more so I can run some more Puma Workers and Threads.</p>
  
  <p><strong>EDIT 2</strong></p>
  
  <p>I ended up having to do this in batches (using offset and limit) and send 5 jobs at a time. The memory usage dropped substantially when I did this. Obviously there was more calls to the controller but each was smaller and faster.</p>