I have a program that is running on multiple machines and which is writing the results to ~/data/machine_name/, each time the results file exceeds 1MB the program moves on and write to a new file. (All machines have the same directory architecture).I want to do an automatic backup of these files in case one of the machines fail. Knowing that the most recent accessed file is the one being written by the program, and hence, I would copy all the old files except this one. The next time the backup job fires, the most recent file omitted in the previous backup will be included and so on..Right now I am using only a simple scp command, which copies even the file being open by the remote machine.Is there anyway to achieve this? a command line utility or a script for such a job?
Copy/Backup all files except the most recent through scp
backup;scp;remote
You can use rsyncbackup utility with --exclude option:/usr/bin/rsync -avz --exclude='$EXCLUDED_DIR_OR_FILE' -e ssh /your/base/folder \user@host_or_ip:/target/machine/path >> /optional/log/file.logYou can determine $EXCLUDED_DIR_OR_FILE variable by finding last updated file. There are several ways to find last updated file in a given directory, one of them is using ls and awk utilities together:ls -lrt | awk '{ f=$NF }; END{ print f }'Please note that this command prints last modified file name and not he full path to the file.