I'm constantly using wget to download many files over 1GB each. I'm getting concerned about how this may be straining my HDD. Can wget's internal buffer size be modified so that it writes to my HDD less often? What about curl?
wget internal buffer size
wget;io;curl
You can use an external bufffer, and pipe the output through it.For example, the program buffer allows you to buffer up to 1GB, and you can specify at which fill level percentage it should start to write :To buffer 10 blocks of 512 kB (5MB) and write out to the file when the buffer is filled to 85%:wget example.com -O- | buffer -s 512k -b 10 -p 85 > ./outfile.txt(may need apt-get install buffer)But, other than directly answering the question, this does not look like something you should need to do manually. Maybe you can configure your file system to cache write operations a little longer before actually writing to the disk? Depends on what problem you want to solve exactly.