mysql 8 information_schema.processlist -> correct approach to cache it?
(I agree that Threads_running would be better.)

As for the Processlist, simply have a single program doing:


Collect processlist
parse and analyze it (there is probably no reason to have each worker do the same processing here)
store the minimal amount of info in some table dedicated to this purpose.  This is probably a single UPDATE statement.  (Not a much heavier TRUNCATE.)
Repeat forever -- Do not repeat every second; do it continually or with a slight delay in the loop.


The reason for not using a cron job:  When the server is too busy, you could get a new copy of the job starting before the first finishes.  It is much better to simply avoid that situation rather than use transactions (or whatever) to deal with it.

MEMORY should be fine, but might not be as fast as InnoDB.  The simple UPDATE is minimally invasive, so the storage engine should not matter much.

No transactions; minimal overhead.

Probably step 2 boils down to the equivalent of SHOW GLOBAL STATUS LIKE 'Threads_running';

If it takes more effort to measure how "busy" the server is than to simple forge ahead, you are wasting your time.

(A side note:  REPLACE is worse than INSERT in this case -- REPLACE does a DELETE, but you have already TRUNCATEd!?)