optimizing a bash script chain - speed suggestions? i am testing an application that i wrote and want to test the solution my algorithm produces to a Monte carlo solution. I use the harddisk a lot and i was wondering if there was a solution that uses writing data to a file a lot less, since it is really slowing the process down.

The solutions are computed on the nodes of a cluster and examined using this script ( that runs on a node): Parameter $1 is an outputfile that the program wrote.

file=$1
script=/home/hefke/ov_paper/scripts
mv $file.out $file.out.old
grep "Overlapscore:" $file.monte &gt; $file.grepped
awk '/./{print $2}' $file.grepped &gt; $file.overlap
print "$script/std_dev.sh $file.overlap &gt; $file.out"
$script/std_dev.sh $file.overlap &gt; $file.out
cat $file.analy &gt;&gt; $file.out
cat "DONE" &gt;&gt; $file.out


Here is the script that collects the data on the main node. Analy and Monte files are my output files.

echo "Processing outputfiles for the mc_stdev_of_ov"
script=/home/hefke/ov_paper/scripts
curdir=`pwd`
folder=filedata
for file in `ls -1 $curdir/temp_output/$folder/*.analy| sed 's/\(.*\)\..*/\1/'|uniq`
do 
    echo $file
    $script/submitter.sh $curdir "processonefile.sh $file.out"
done
echo "$file.out now contains what stdtev spat out."
cat $curdir/temp_output/$folder/*.out &gt;&gt; $curdir/temp_output/tmp.out 
awk -f keys.awk $curdir/temp_output/tmp.out &gt;&gt; table.out
cat table.out


How can i optimize this procedure for speed?
It's not related, but please don't mind if I use an "answer" to just comment : it seems I can't comment, maybe because I don't have enough points yet to do so... 

Tarrasch, if you still use csh for your shell, please do not script in it.

Please read: http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

Use instead sh, bash (or even ksh). And better to stick to sh-only because that's what's all unix system rely on (and rc scripts, for example, are based on).