title: Index count on scan + filter expression -> overcoming ProvisionedThroughputExceededException
body: <p>On a 10 million records dynamo db where each item has an 'epoch' time stamp attribute I'm trying to count the items between a range of epochs.
  The dynamo table has a provisioned read capacity units of 1000. Each item is 5~7Kb.</p>
  
  <p>The code:</p>
  
  <pre><code>from boto3.session import Session
  from boto3.dynamodb.conditions import Attr
  
  START_EPOCH = 1443657600000
  END_EPOCH =   1443744000000
  TOTAL_ITEMS_TO_SCAN = 1000000
  F_EXP = Attr('epoch').gt(START_EPOCH) &amp; Attr('epoch').lt(END_EPOCH)
  
  session = Session(aws_access_key_id='access_key',
                    aws_secret_access_key='secret_key',
                    region_name='region')
  resource = session.resource('dynamodb')
  table = resource.Table('table name')
  
  def scan_func(last_key,counter,scanned):
      if last_key:
          result = table.scan(FilterExpression=F_EXP,
                              Select='COUNT',
                              ExclusiveStartKey=last_key)
      else:
          result = table.scan(FilterExpression=F_EXP,
                              Select='COUNT')
      counter += result['Count']
      scanned += result['ScannedCount']
      print "Current items found {} from {} scanned".format(counter, scanned)
  
      if counter &lt; TOTAL_ITEMS_TO_SCAN:
          scan_func(result['LastEvaluatedKey'], counter, scanned)
  
      print 'total items founs : {}, from {} scanned'.format(counter,scanned)
  
  scan_func(None, 0, 0)
  </code></pre>
  
  <p>Even when running the scan using segments, on average after few iterations I get the following response:</p>
  
  <blockquote>
  <pre><code>botocore.exceptions.ClientError: An error occurred
  (ProvisionedThroughputExceededException) when calling the Scan operation: 
  The level of configured provisioned throughput for the table was exceeded. 
  Consider increasing your provisioning level with the UpdateTable API
  </code></pre>
  </blockquote>
  
  <p>The best result I've got so far is:</p>
  
  <blockquote>
  <pre><code>Current items found 16 from 3245 scanned
  </code></pre>
  </blockquote>
  
  <p>I also tried to induce a sleep of 2 seconds between each iteration to give the db space to recover and free provisioned resources, and did not work either.
  Also tried to increase by 3 times the provisioned resources to 3,000 instead of 1,000 and it did some more iterations but eventually stopped.</p>
  
  <p>Any ideas on how to make this work?
  Other alternatives without increasing the read capacity of the table?</p>