I have a task that processes a list of files on stdin.  The start-up time of the program is substantial, and the amount of time each file takes varies widely.  I want to spawn a substantial number of these processes, then dispatch work to whichever ones are not busy.  There are several different commandline tools that almost do what I want, I've narrowed it down to two almost working options:find . -type f | split -n r/24 -u --filter=myjobfind . -type f | parallel --pipe -u -l 1 myjobThe problem is that split does a pure round-robin, so one of the processes gets behind and stays behind, delaying the completion of the entire operation; while parallel wants to spawn one process per N lines or bytes of input and I wind up spending way too much time on startup overhead.Is there something like this that will re-use the processes and feed lines to whichever processes have unblocked stdins?
Spreading stdin to parallel processes
xargs;parallel;split