This is a simple tool to parallelize execution of multiple tasks in Bash, since env_parallel can't handle the environment size (or something of the sort - dies for mysterious reasons.) I'd love to hear your critique. ## Simple parallel executor. Takes a list of commands and executes each in its own child process. Assumes commands are distinctly named.# Returns with 0 if all are successful. Otherwise, prints out their stdout/stderr outputs and returns with 1.function simple_par () {    local TMP_FILE=${WRITABLE_DIR}/simple_par_$(uuidgen)    touch $TMP_FILE    for prog in $@    do        local PROG_TMP=${TMP_FILE}_${prog}_$(uuidgen)        (            ( $prog 1>$PROG_TMP 2>&1 )            echo $? >> $TMP_FILE        ) &    done    while true    do        RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')        if [[ $RESULT_COUNT -eq $# ]]; then            if [[ $(cat $TMP_FILE) == *1* ]]; then                echo Failed at least one command                for prog in $@                do                    echo OUTPUT FOR: $prog                    cat ${TMP_FILE}_${prog}_*                    rm ${TMP_FILE}_${prog}_*                done                rm $TMP_FILE                return 1            else                return 0            fi        fi        sleep 5    done}
Simple Bash Parallel Tool (env_parallel dies on big env)
bash;multiprocessing
A couple of things could be written better, let's go from top to bottom.It's recommended to double-quote variables when used as paths,to avoid globbing and word-splitting.Even if you're certain that $TMP_FILE won't have any troublesome characters, it's a good practice to double-quote anyway:touch $TMP_FILESimilarly, you should double-quote $prog in $prog 1>$PROG_TMP 2>&1.You can simplify for arg in $@; do loop as for arg; do.In Bash, instead of prog 1> somewhere 2>&1 you can write prog &> somewhere.Instead of this:RESULT_COUNT=$(wc -l ${TMP_FILE} | awk '{ print $1 }')You could write simpler without awk like this:RESULT_COUNT=$(wc -l < $TMP_FILE)The trick is that when wc is used with stdin,it won't print a file name, only the number.Although RESULT_COUNT will have some whitespace padding,if you use it in [ ... ] without double-quoting,it will behave as expected.Notice again that I double-quoted $TMP_FILE.Don't use -eq in [[ ... ]], use == or = instead.This is pretty ugly:if [[ $(cat $TMP_FILE) == *1* ]]; thenA better way to achieve the same thing is using grep:if grep -q 1 $TMP_FILE; then