title: Batch insert/update from one table to another within the Postgres database using Psycopg2
body: <p>I am inserting/updating 5 million rows into the target table from source table within Postgres, In order to run my update and insert queries I am using psycopg2 python package and doing something very plain as opening the connection and then opening the cursor ,and running</p>
  <p>cursor.execute(update_query)</p>
  <p>cursor.execute(insert_query)</p>
  <p>It does work just fine .However from the documentation I understand that psycopg2 provides certain (execute_many, execute_batch) methods to reduce network roundtrip, can I take the advantage of these methods while updating, inserting the rows <strong>within</strong> the database as well ?</p>
  <p><strong>Queries used :-</strong></p>
  <p><strong>Update:-</strong></p>
  <pre><code>UPDATE target_tbl tgt
          set descr = stage.descr,
          prod_name = stage.prod_name,
          item_name = stage.item_name,
          url       = stage.url,
          col1_name = stage.col1_name,
          col2_name = stage.col2_name,
          col3_name = stage.col3_name,
          col4_name = stage.col4_name,
          col5_name = stage.col5_name,
          col6_name = stage.col6_name,
          col7_name = stage.col7_name,
          col8_name = stage.col8_name,
          flag      = stage.flag
      from tbl1 stage
      where 
      tgt.col1 = stage.col1
      and tgt.col2 = stage.col2
      and coalesce(tgt.col3, 'col3'::text) = coalesce(stage.col3, 'col3'::text)
      and coalesce(tgt.col4, 'col4'::text) = coalesce(stage.col4, 'col4'::text);
      
  </code></pre>
  <p><strong>Insert:-</strong></p>
  <pre><code>Insert into tgt
      select 
      stage.col1,
      stage.col2,
      stage.col3,
      stage.col4
      stage.prod_name,
      stage.item_name,
      stage.url,
      stage.col1_name,
      stage.col2_name,
      stage.col3_name,
      stage.col4_name,
      stage.col5_name,
      stage.col6_name,
      stage.col7_name,
      stage.col8_name,
      stage.flag
      from tbl1 stage
      where NOT EXISTS (
      select from tgt where
      tgt.col1 = stage.col1
      and tgt.col2 = stage.col2
      and coalesce(tgt.col3, 'col3'::text) = coalesce(stage.col3, 'col3'::text)
      and coalesce(tgt.col4, 'col4'::text) = coalesce(stage.col4, 'col4'::text)
      ) ;
  </code></pre>