title: How to control concurrency of a subdag or taskgroup in Airflow?
body: <p>I have a simple ETL workflow composed of three tasks for each table I want to extract :</p>
  <p><code>1_extract_to_tmp</code> &gt;&gt; <code>2_push_to_s3</code> &gt;&gt; <code>3_delete_tmp</code></p>
  <p>As I want to reproduce the same steps for multiple tables, I was thinking of grouping these task in a TaskGroup or a subDAG and instanciate it dynamically for each table to extract. My final DAG will look like this :
  <a href="https://i.stack.imgur.com/pU2L3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pU2L3.png" alt="My DAG" /></a></p>
  <p>From what I read, the TaskGroup is the prefered solution now to do that.</p>
  <p>The problem is that I also need to control how much extractions are done in parallel because I don't have enough local disk space for all data and apparently the subDag operator does not comply to pool setting and the TaskGroup does not have one.</p>
  <p>Do you know a way to achieve this ?</p>
  <p>Am I doing it wrong ?</p>
  <p>Maybe my DAG should not be designed like that.</p>