title: Batching when using ActiveRecord::Base.connection.execute
body: <p>I am busy writing an migration that will allow us to move our yamler from Syck to Psych and finally upgrade our project to ruby 2. This migration is going to be seriously resource intensive though so I am going to need to use chunking.</p>
  
  <p>I wrote the following method to confirm that the result of the migration I plan to use produces the expected result and can be done without down time. To avoid Active record performing the serialization automatically I needed to use <code>ActiveRecord::Base.connection.execute</code></p>
  
  <p>My method that describes the transformation is as follows </p>
  
  <pre><code> def show_summary(table, column_name)
    a = ActiveRecord::Base.connection.execute &lt;&lt;-SQL
     SELECT id, #{column_name} FROM #{table}
    SQL
    all_rows = a.to_a; ""
    problem_rows = all_rows.select do |row|
      original_string = Syck.dump(Syck.load(row[1]))
      orginal_object = Syck.load(original_string)
  
      new_string = Psych.dump(orginal_object)
      new_object = Syck.load(new_string)
  
      Syck.dump(new_object) != original_string rescue true
    end
  
  problem_rows.map do |row|
    old_string = Syck.dump(Syck.load(row[1]))
    new_string = Psych.dump(Syck.load(old_string)) rescue "Parse failure"
    roundtrip_string = begin
      Syck.dump(Syck.load(new_string))
    rescue =&gt; e
      e.message
    end
  
    new_row = {}
    new_row[:id] = row[0]
    new_row[:original_encoding] = old_string
    new_row[:new_encoding] = roundtrip_string
    new_row
    end
  end
  </code></pre>
  
  <p>How can you use batching when making use of  <code>ActiveRecord::Base.connection.execute</code> ?</p>
  
  <p>For completeness my update function is as follows </p>
  
  <pre><code>  # Migrate the given serialized YAML column from Syck to Psych
    # (if any).
    def migrate_to_psych(table, column)
      table_name = ActiveRecord::Base.connection.quote_table_name(table)
  
      column_name = ActiveRecord::Base.connection.quote_column_name(column)
  
      fetch_data(table_name, column_name).each do |row|
        transformed = ::Psych.dump(convert(Syck.load(row[column])))
  
        ActiveRecord::Base.connection.execute &lt;&lt;-SQL
           UPDATE #{table_name}
           SET #{column_name} = #{ActiveRecord::Base.connection.quote(transformed)}
           WHERE id = #{row['id']};
        SQL
      end
    end
  
    def fetch_data(table_name, column_name)
      ActiveRecord::Base.connection.select_all &lt;&lt;-SQL
         SELECT id, #{column_name}
         FROM #{table_name}
         WHERE #{column_name} LIKE '---%'
      SQL
    end
  </code></pre>
  
  <p>Which I got from <a href="http://fossies.org/linux/openproject/db/migrate/migration_utils/legacy_yamler.rb" rel="nofollow">http://fossies.org/linux/openproject/db/migrate/migration_utils/legacy_yamler.rb</a>   </p>