title: how to speed up a query with partitionkey in azure table storage
body: <p><strong>How do we increase the speed of this query?</strong></p>
  
  <p>We have approximately <strong>100 consumer</strong>s within the span of <code>1-2 minutes</code> executing the following query. Each one of these runs represents 1 run of a consumption function. </p>
  
  <pre><code>        TableQuery&lt;T&gt; treanslationsQuery = new TableQuery&lt;T&gt;()
           .Where(
            TableQuery.CombineFilters(
              TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, sourceDestinationPartitionKey)
             , TableOperators.Or,
              TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, anySourceDestinationPartitionKey)
            )
           );
  </code></pre>
  
  <p>This query will yield approximately <strong>5000 results.</strong> </p>
  
  <p><strong>Full code:</strong></p>
  
  <pre><code>    public static async Task&lt;IEnumerable&lt;T&gt;&gt; ExecuteQueryAsync&lt;T&gt;(this CloudTable table, TableQuery&lt;T&gt; query) where T : ITableEntity, new()
      {
          var items = new List&lt;T&gt;();
          TableContinuationToken token = null;
  
          do
          {
              TableQuerySegment&lt;T&gt; seg = await table.ExecuteQuerySegmentedAsync(query, token);
              token = seg.ContinuationToken;
              items.AddRange(seg);
          } while (token != null);
  
          return items;
      }
  
      public static IEnumerable&lt;Translation&gt; Get&lt;T&gt;(string sourceParty, string destinationParty, string wildcardSourceParty, string tableName) where T : ITableEntity, new()
      {
          var acc = CloudStorageAccount.Parse(Environment.GetEnvironmentVariable("conn"));
          var tableClient = acc.CreateCloudTableClient();
          var table = tableClient.GetTableReference(Environment.GetEnvironmentVariable("TableCache"));
          var sourceDestinationPartitionKey = $"{sourceParty.ToLowerTrim()}-{destinationParty.ToLowerTrim()}";
          var anySourceDestinationPartitionKey = $"{wildcardSourceParty}-{destinationParty.ToLowerTrim()}";
  
          TableQuery&lt;T&gt; treanslationsQuery = new TableQuery&lt;T&gt;()
           .Where(
            TableQuery.CombineFilters(
              TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, sourceDestinationPartitionKey)
             , TableOperators.Or,
              TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, anySourceDestinationPartitionKey)
            )
           );
  
          var over1000Results = table.ExecuteQueryAsync(treanslationsQuery).Result.Cast&lt;Translation&gt;();
          return over1000Results.Where(x =&gt; x.expireAt &gt; DateTime.Now)
                             .Where(x =&gt; x.effectiveAt &lt; DateTime.Now);
      }
  </code></pre>
  
  <p>During these executions, when there are 100 consumers, as you can see the requests will cluster and form spikes:</p>
  
  <p><a href="https://i.stack.imgur.com/s9DkA.png" rel="noreferrer"><img src="https://i.stack.imgur.com/s9DkA.png" alt="enter image description here"></a></p>
  
  <p>During these spikes, the requests often take over 1 minute:</p>
  
  <p><a href="https://i.stack.imgur.com/N1S9Q.png" rel="noreferrer"><img src="https://i.stack.imgur.com/N1S9Q.png" alt="enter image description here"></a></p>
  
  <p><strong>How do we increase the speed of this query?</strong></p>