title: Asyncio: How to handle multiple open files OS error
body: <p>I am trying to run ~500 async subprocesses. I am passing the files as list <strong>p_coros</strong> in the main function below.</p>
  
  <pre><code>async def run_check(shell_command):
      p = await asyncio.create_subprocess_shell(shell_command,
                      stdin=PIPE, stdout=PIPE, stderr=STDOUT)
      fut = p.communicate()
      try:
          pcap_run = await asyncio.wait_for(fut, timeout=5)
      except asyncio.TimeoutError:
          p.kill()
          await p.communicate()
  
  def get_coros():
      for pcap_loc in print_dir_cointent():
          for pcap_check in get_pcap_executables():
              tmp_coro = (run_check('{args}'
              .format(e=sys.executable, args=args)))
              if tmp_coro != False:
                  coros.append(tmp_coro)
       return coros
  
  async def main(self):
      ## Here p_coros has over 500 files
      p_coros = get_coros()
      for f in asyncio.as_completed(p_coros):
          res = await f
  
  
  
  
  loop = asyncio.get_event_loop()
  loop.run_until_complete(get_coros())
  loop.close()
  </code></pre>
  
  <p>The issue I think here is <strong>asyncio.as_completed</strong>, as it's trying to open all files parallelly, because if I remove <strong>asyncio.as_completed</strong> it works properly but takes a lot of time.
  I wanted to handle the open file issue <em>OSError(24, 'Too many open files')</em> without losing much time.</p>
  
  <p>Logs:</p>
  
  <pre><code>Exception ignored when trying to write to the signal wakeup fd:
  BlockingIOError: [Errno 11] Resource temporarily unavailable
  
  ERROR:asyncio:Task was destroyed but it is pending!
  task: &lt;Task pending coro=&lt;ClassificationCheck.run_check() running at ./regression.py:74&gt; wait_for=&lt;Future finished exception=RuntimeError('Event loop is closed',)&gt; cb=[as_completed.&lt;locals&gt;._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:478]&gt;
  </code></pre>
  
  <p>Traceback:</p>
  
  <pre><code>Traceback (most recent call last):
    File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
      result = coro.send(None)
    File "./regression.py", line 74, in run_check
      stdin=PIPE, stdout=PIPE, stderr=STDOUT)
    File "/usr/lib/python3.5/asyncio/subprocess.py", line 197, in create_subprocess_shell
      stderr=stderr, **kwds)
    File "/usr/lib/python3.5/asyncio/base_events.py", line 1049, in subprocess_shell
      protocol, cmd, True, stdin, stdout, stderr, bufsize, **kwargs)
    File "/usr/lib/python3.5/asyncio/unix_events.py", line 184, in _make_subprocess_transport
      **kwargs)
    File "/usr/lib/python3.5/asyncio/base_subprocess.py", line 40, in __init__
      stderr=stderr, bufsize=bufsize, **kwargs)
    File "/usr/lib/python3.5/asyncio/unix_events.py", line 640, in _start
      stdin, stdin_w = self._loop._socketpair()
    File "/usr/lib/python3.5/asyncio/unix_events.py", line 53, in _socketpair
      return socket.socketpair()
    File "/usr/lib/python3.5/socket.py", line 478, in socketpair
      a, b = _socket.socketpair(family, type, proto)
  OSError: [Errno 24] Too many open files
  ERROR:asyncio:Task exception was never retrieved
  future: &lt;Task finished coro=&lt;ClassificationCheck.run_check() done, defined at ./regression.py:72&gt; exception=OSError(24, 'Too many open files')&gt;
  </code></pre>