title: Writing data to a csv file line by line in python
body: <p>I would like to extract data with cur.fetchmany(size) to write it into a csv file and finally using cur.copy_from() to copy that data to another table. I am aware that there are other solutions but my way has to be like data-csv-table. I have more than 100k rows and am trying to iterate cur.fetchmany() method to do it part by part. In my code youc an see I defined a function to write csv and iterate data. I added the error. I looked for the errpr but couldn't find solution which suits my question. My question is: Why do I get that error and what should I do to fix it as I mentioned below?</p>
  
  <pre class="lang-py prettyprint-override"><code>import psycopg2
  import csv
  
  connection = psycopg2.connect(user = 'user', password = 'password',host='192.850.1.45', database = 'test', port = 5432)
  
  cur = connection.cursor()
  print('Database connected successfully')
  cur.execute('Select cust_ord_id::bigint, cust_ord_item_id::bigint from dwf_sales limit 10000')
  
  def csvyaz(num):
      with open('C:/Users/oguz.catal/Desktop/newfile.csv', 'wb', ) as filecsv:
          writer = csv.writer(filecsv, delimiter=',', quotechar='"')
          writer.writerow(num)
      return 'C:/Users/oguz.catal/Desktop/newfile.csv'
  
  def iter_row(cursor, size=1000):
      while True:
          rows = cursor.fetchmany(size)
          num = list(sum(rows, ()))
          if rows is None:
              break
          for row in rows:
              yield row
          print(num)
          csvyaz(num)
          t = open('C:/Users/oguz.catal/Desktop/newfile.csv', 'r+')
          cur.copy_from(t,'oguz_deneme',sep=',', null='null', columns=('cust_ord_id', 'cust_ord_item_id'))
  
  for row in iter_row(cur, 1000):
      print(row)
  </code></pre>
  
  <p>I get this error: </p>
  
  <blockquote>
    <p>'Traceback (most recent call last):  File
    "C:/Users/oguz.catal/PycharmProjects/untitled/fetchmannny.py", line
    34, in   for row in iter_row(cur, 1000):  File
    "C:/Users/oguz.catal/PycharmProjects/untitled/fetchmannny.py", line
    31, in iter_row  cur.copy_from(t,'oguz_deneme',sep=',', null='null',
    columns=('cust_ord_id', 'cust_ord_item_id')) 
    psycopg2.errors.BadCopyFileFormat: extra data after last expected
    column  CONTEXT:  COPY oguz_deneme, line 1:
    "6800008477,4400033838,6800008477,4400033834,6800008477,4400033832,6800008477,4400033827,6800008732,4..."'</p>
  </blockquote>
  
  <p>My csv is like this: </p>
  
  <pre><code>6800008477  4400033838  6800008477  4400033834  6800008477  4400033832  6800008477  4400033827  6800008732  4400034887
  </code></pre>
  
  <p>But as far as I undersand it should be like this to use copy_from() properly;</p>
  
  <pre><code>6800008477  4400033838  
  6800008477  4400033834  
  6800008477  4400033832  
  6800008477  4400033827  
  6800008732  4400034887
  </code></pre>