I have a command-line script that performs an API call and updates a database with the results.I have a limit of 5 API calls per second with the API provider. The script takes more than 0.2 seconds to execute.If I run the command sequentially, it will not run fast enough and I will only be making 1 or 2 API calls per second.If I run the command sequentially, but simultaneously from several terminals, I might exceed the 5 calls / second limit.If there a way to orchestrate threads so that my command-line script is executed almost exactly 5 times per second?For example something that would run with 5 or 10 threads, and no thread would execute the script if a previous thread has executed it less than 200ms ago.
How to run a command at an average of 5 times per second?
command line;multithreading
I've used Stphane Chazelas' pv-based solution for some time, but found out that it exited randomly (and silently) after some time, anywhere from a few minutes to a few hours. -- Edit: The reason was that my PHP script occasionally died because of a max execution time exceeded, exiting with status 255.So I decided to write a simple command-line tool that does exactly what I need.Achieving my original goal is as simple as:./parallel.phar 5 20 ./my-command-line-scriptIt starts almost exactly 5 commands per second, unless there are already 20 concurrent processes, in which case it skips the next execution(s) until a slot becomes available.This tool is not sensitive to a status 255 exit.