I am mirroring a website using wget.On the server, there is a dump of the find / command, which includes filenames in the form :/dir1//dir1/page1.txt/dir1/page2.txt...The mirrorring is very large (100GB+) and I would like to compare the local output of find to the text file containing the server-side dump, so as to have an idea of how many files are left.I was thinking of using a grep -v ... or awk but I'm not quite sure how one would go about doing that.Can anyone help me please?Thanks !
Find + Grep Recursive Search
grep;find;wget;mirror
You can do this with comm (using bash process substitution):comm -13 <(find / | sort) <(sort server_dump)This will show the files which are unique to the server. For files that are unique to the local system:comm -23 <(find / | sort) <(sort server_dump)You may also wan to add the -xdev option to find to stop it from comparing files on other filesystems, such as /proc and /sys.