title: How to download large data from a SQL table and consecutively save into csv by fetching 1000 or so records at once
body: <p>I have a SQL table consisting of 10 million rows and lot of columns, The table size when queried is around 44 GB.</p>
  
  <p>However I am trying to fetch only 3 columns from this table and save it into csv / load into dataframe the python keeps running forever.
  i.e</p>
  
  <pre><code> pd.read_sql("select a,b,c from table") is taking more than 1 hour and not returning data
  </code></pre>
  
  <pre>How to achieve this? Can I load this entire data in dataframe at once is that a viable option.
  After this I should be able to perform some data manipulations on these rows.
  
  2. OR should I download this to csv and read this data part by part to in-memory.</pre>
  
  <p>If its 2. How to code for 2?</p>
  
  <p>Code tried for 2 so far is : </p>
  
  <pre><code>   def iter_row(cursor, size=10):
  while True:
      rows = cursor.fetchmany(size)
      if not rows:
          break
      for row in rows:
          yield row
  
    def query_with_fetchmany():
  
      cursor.execute("SELECT * FROM books")
  
      for row in iter_row(cursor, 10):
          print(row)
      cursor.close()
  </code></pre>