High CPU utilisation for Logs Manager shell script I have written a shell script to manage the tcpdump pcap logs and syslog files in my Linux board, so as to maintain the disk usage to maximum of 70%. 

The script checks for the disk usage every minute and as soon as the disk goes higher than 70%, it deletes all the oldest files that are not in use until the disk usage goes back to 70%.

#!/bin/ash

#Gives the file that is currently in use by the process
logmgr_get_current_file_name() {
    PROC_NAME=$1
    instance=$2

    pid=$(ps | grep $PROC_NAME | grep -v grep | awk 'NR=='$instance'{print $1}')

    fds=$(ls /proc/$pid/fd)

    for fd in $fds;
    do
        last=$fd
    done

    current_file=$(ls /proc/$pid/fd/$last -l | awk 'BEGIN {FS="-&gt;"} {printf $2}')
}

#checks whether the file we are deleting is in use by the process 
logmgr_is_file_in_use() {
    num_of_instances=1
    logmgr_get_current_file_name $1 $num_of_instances
    if [ $current_file = $2 ]; then
        return 1
    else
       return  0
    fi
}

# iterate through all possible instances of tcpdump
# for each instance .. get the file in use
# check if [ $file_in_use == $to_delete ] ; then
# return 1
# else
# return 0
logmgr_get_files_in_use() {
    i=1
    is_file_inuse=0

    num_of_instances=$(ps | grep tcpdump| grep -v grep | wc -l)

    while [ $i -le $num_of_instances ];
    do
        logmgr_get_current_file_name $1 $i
        if [ "$current_file" = "$2" ]; then
            is_file_inuse=1
            break
        fi
                i=`expr $i + 1`
        done
        return $is_file_inuse
}

logmgr_delete_oldest() {

    oldest=$(ls -lht $2 | awk 'END{print $9}')
    to_delete=$2$oldest

    if [ "$1" != "tcpdump" ] ; then
        logmgr_is_file_in_use $1 $to_delete
        ret_val=$?
    else
        logmgr_get_files_in_use $1 $to_delete "tcpdump"
        ret_val=$?
    fi

    #if the file is not in use, then delete it
    if [ $ret_val -eq 0 ]; then
        echo "$to_delete deleted"
        rm -fr $to_delete
        sleep $cpu_relaxation_secs
    fi

}


logmgr_get_disk_usage() {
    diskusage=$(df -h $1 | awk 'END {print $5}' | awk 'BEGIN {FS="%"} {print $1}')
    echo $diskusage
}

    ##requires repeated execution######
logmgr_run() {

    sleep_secs=0

    while [ 1 ] ;
    do
        syslog_dir_is_ok=0
        tcpdump_dir_is_ok=0

        syslog_disk_usage=`logmgr_get_disk_usage $syslog_dir`
        tcpdump_disk_usage=`logmgr_get_disk_usage $tcpdump_dir`

        if [ $syslog_disk_usage -le $NORMAL_DISK_USAGE ] ; then
            syslog_dir_is_ok=1
        fi

        if [ $tcpdump_disk_usage -le $NORMAL_DISK_USAGE ] ; then
            tcpdump_dir_is_ok=1
        fi

        if [ $syslog_dir_is_ok -eq 0 ] &amp;&amp; [ $tcpdump_dir_is_ok -eq 0 ] ; then
            break
        fi

        if [ $syslog_dir_is_ok -eq 0 ] ; then
            num_files=`ls -lht $syslog_dir | wc -l`
            logmgr_delete_oldest "syslogd" $syslog_dir $num_files
        fi

        if [ $tcpdump_dir_is_ok -eq 0 ] ; then
            num_files=`ls -lht $tcpdump_dir | wc -l`
            logmgr_delete_oldest "tcpdump" $tcpdump_dir $num_files
        fi
    done
}

##Default values##

max_file_safe=100
cpu_relaxation_secs=1
timer_secs=60   # one minute
KEEP_CHECKING=1
NORMAL_DISK_USAGE=20

# read the configuration file
syslog_dir=`uci get syslog.slog.directory`
syslog_dir=`dirname $syslog_dir`"/"`basename $syslog_dir`"/"
tcpdump_dir=`uci get tcpdump.tcpdump.log_dir`
tcpdump_dir=`dirname $tcpdump_dir`"/"`basename $tcpdump_dir`"/"

#while [ $KEEP_CHECKING -eq 1 ]
while :
do
    logmgr_run
    sleep $timer_secs
done


The script is sequential and works perfectly fine but the CPU utilisation gets higher to about 75% while running the script. I tried to remove all the redundancies in script but still not successful in achieving the desired result.

I am sure there are ways with which I can achieve this, but am not sure how. Please suggest some ways to optimise this script and reduce my CPU utilisation.
If your CPU is rising to 75% and staying there (not bouncing up and down), then it suggest that your script is spinning and not hitting any of your sleep statements.  There's a few things that may be worth looking into.


The while loop in logmgr_run keeps going until enough files have been cleared out.  If this doesn't happen, then it'll never get to the 60 second sleep.
logmgr_delete_oldest will only sleep if it deletes a file, which only happens if the file isn't in use.


I haven't tested it, and it has been a while since I did script programming, but it looks to me like your algorithm breaks down if the oldest file in either folder is being used at the same time that you breach the usage threshold.  It can't delete the oldest file, so it just keeps trying over and over again without succeeding or sleeping.

Rather than having a loop that keeps trying to remove the last file when the threshold is breached it might be better to calculate the amount of files that need to be removed in order to come back under the threshold and try to remove several files in one go / or to skip files if they are in use and move on to the next file rather than stopping and trying the same file again next time.

It may also be worthwhile extending the cleanup to clear a bit more than you need, so that you don't immediately trigger another cleanup next time you go through the loop.  So, for example if you breach 70% usage, clear down the files to 65% or 60% so that there is some growth space.

Your script also has some declared an unused variable, which you should probably either be using (or remove):

max_file_safe=100