title: PostgreSQL query taking too long via Python
body: <pre><code>#!/usr/bin/env python
  import pika
  
  
  def doQuery( conn, i ) :
      cur = conn.cursor()
  
      cur.execute("SELECT * FROM table OFFSET %s LIMIT 100000", (i,))
  
  
      return cur.fetchall()
  
  
  print "Using psycopg2"
  import psycopg2
  myConnection = psycopg2.connect( host=hostname, user=username, 
  password=password, dbname=database )
  
  connection = 
  pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
  channel = connection.channel()
  
  channel.queue_declare(queue='task_queue2')
  
  endloop = False
  i = 1
  while True:
  
    results = doQuery( myConnection, i )
  
  
  
      j = 0
      while j &lt; 10000:
  
        try:
          results[j][-1]
        except:
          endloop = True
        break
  
  
        message = str(results[j][-1]).encode("hex")
  
        channel.basic_publish(exchange='',
                  routing_key='task_queue2',
                  body=message
                  #properties=pika.BasicProperties(
                      #delivery_mode = 2, # make message persistent
                  )#)
  
  
  
         j = j + 1
  
  # if i % 10000 == 0:
  #   print i
  
    if endloop == False:
      break        
  
    i = i + 10000
  </code></pre>
  
  <p>The SQL query is taking too long to execute when <code>i</code> gets to 100,000,000, but I have about two <em>billion</em> entries I need to put into the queue. Anyone know of a more efficient SQL query that I can run so that I can get all those two billion into the queue faster?</p>