title: How to load data into pandas from a large database?
body: <p>I have a postgres database which contains time series data.The size of the database is around 1 GB.Currently to read data, this is what I do</p>
  
  <pre><code>import psycopg2
  import pandas as pd
  import pandas.io.sql as psql
  
  conn = psycopg2.connect(database="metrics", user="*******", password="*******", host="localhost", port="5432")
  cur = conn.cursor()
  df = psql.read_sql("Select * from timeseries", conn)
  print(df)
  </code></pre>
  
  <p>But this loads the entire data into the memory.Now I am aware of techniques where the database can be dumped to a csv file and then the csv file can be read in chunks as suggested here <a href="https://stackoverflow.com/questions/25962114/how-to-read-a-6-gb-csv-file-with-pandas">How to read a 6 GB csv file with pandas</a></p>
  
  <p>But for me that is not an option since the database will be continously changing and I need to read it on the fly.Is there any technique to read the database content maybe in chunks or use any third party libraries?</p>