title: ensuring no more than N instances are scaled out for a specific queue
body: <p>My function is sending a payload to different sftp servers. Those servers are limited in how many connections they can accept. </p>
  
  <p>I need a solution to throttle our connections to those servers. </p>
  
  <p>The function is triggered by storage queues, and the first draft of the design is:</p>
  
  <p><a href="https://i.stack.imgur.com/fkGWX.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fkGWX.png" alt="enter image description here"></a></p>
  
  <p>I then learned that you can only have 1 trigger per function, which led me to sandwhich another aggregating queue:</p>
  
  <p><a href="https://i.stack.imgur.com/VHmnP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/VHmnP.png" alt="enter image description here"></a></p>
  
  <p>I can set the <a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue#host-json" rel="nofollow noreferrer">batchSize/newBatchThreshold</a> on the originating queues, but I'm not certain this will work because the originating queues will not be aware of when to push messages to the <em>aggregate queue</em>.</p>
  
  <ol>
  <li>I need the function to not scale out to more than N instances for all messages from queue X, since the sftp server X will not accept more than N connections.</li>
  <li>Additionally, I need the function to scale out to no more than M instances for all messages from queue Y, since the sftp server Y will not accept more than M connections. </li>
  </ol>
  
  <p><strong>The total instances would be M + N for the above scenario.</strong> </p>
  
  <p>How do we adjust our design in order to fit these requirements?</p>