I have a list of thousands of URLs. I want to get a health check (healt.php) with an http request.

This is my problem:

I've wrote an application in node. It makes the requests in a pooled way. I use a variable to control how many concurrent connections I open. 300, ie.
One by one, each request is so fast, no more than 500ms.

But when I run the application, the result is:

```
$ node agent.js

200ms   url1.tld
250ms   url4.tld
400ms   url2.tld
530ms   url8.tld
800ms   url3.tld
...
2300ms  urlN.tld
...
30120ms urlM.tld

```

It seems that there is a limit in concurrency. When I execute

```
$ ps axo nlwp,cmd | grep node

```

The result is:

```
6 node agent.js

```

There are 6 threads to manage all concurrent connections. I found an evn variable to control concurrency in node: UV\_THREADPOOL\_SIZE

```
$ UV_THREADPOOL_SIZE=300 node agent.js

200ms   url1.tld
210ms   url4.tld
220ms   url2.tld
240ms   url8.tld
400ms   url3.tld
...
800ms  urlN.tld
...
1010ms urlM.tld

```

The problem is still there, but the results are much better. With the ps command:

```
$ ps axo nlwp,cmd | grep node

132 node agent.js

```

Next step: Looking in the source code of node, I've found a constant in deps/uv/src/unix/threadpool.c:

```
#define MAX_THREADPOOL_SIZE 128

```

Ok. I've changed that value to 2048, compiled and installed node and run once the command

```
$ UV_THREADPOOL_SIZE=300 node agent.js

```

All seems ok. Response times are not incrementing gradually. But when I try with a bigger concurrency number the problema appears. But this time it's not related to the number of threads, because with the ps command I see there are enough of them.

I tried to write the same application in golang, but the results are the same. The time is increasing gradually.

So, my question is: Where is the concurrence limit? memory and cpu load and bandwith are not out of bounds. And I tuned sysctl.conf and limits.conf to avoid some limits (files, ports, memory, ...).