How to handle MySQL database connections/pooling from Lambda -> RDS with Node
Spinning off thousands of processes to do tiny pieces of a task is simple.  But inefficient.  "Processes" are heavyweight.  Threads are somewhat heavy.  Thousands of connections to MySQL is possible, but not all at the same time.

In any multi-process or multi-threaded environment, having "too many" will slow things down.  This is because the OS (or some entity) is doing a lot of work to share the inadequate resources.  MySQL, for example, can handle hundreds of idle connections, but bogs down with more than a few dozen active connections.  When that is exceeded, it is best to go up the chain to the client, and throttle how often it is spawning new connections.

I am not familiar with the other products you mention, but I would recommend keeping simultaneous activity down to dozens, not hundreds.

Or find a way. in your application, to "iterate" instead of "recurse".  Meanwhile, keep in mind that MySQL is happier to handle "vectors" (tables) of data than to handle individual rows.  Maybe you can push some of what is "parallel" in the app down into MySQL as "vectors"?  Win-win:  Fewer connections; more efficiency in MySQL.