Very hard to sync Core Client from scratch Recently I started running a Core 15.1 node with the blockchain on an external hard drive, and found it very hard to fully sync the blockchain.
The problem I keep running into is that (in about 90% of my attempts) the sync does not complete. A DB corruption occurs at some point (usually pretty late in the game, maybe 36 hours into it).

I did some digging in forums and saw that I am not the first user to notice this problem. Peter Wuille mentioned that he does a full sync on like a weekly basis and it always works, and he thus suspects that these type of issues are due to sub optimal hardware.

That seemed very reasonable at first because my initial attempt was done using a pretty old 400gig pocket hard drive. So I switched hard drives. 
By now, about 2 months later, I have tried to sync the blockchain from scratch about 20 or 30 times and only twice managed to complete the process.
I did that on 4 different types of hard drives, including a brand new 1T Western Digital.

One may say that all 4 hard drives I used are "suboptimal", but I would object to that because


After the sync does complete, the node runs just fine. 
I think it is
desirable to be able to run a full node on a raspberry pi with a
pocket size hard drive (note that I run the node on a raspberry pi
but use my linux desktop to do the initial sync)


Now the issue I have here is not so much the crash during sync, but the fact that when I restart the client after the crash the only option is to basically start from scratch (If I understand correctly, the client is not necessarily downloading everything from scratch but that is not my point) and sync again from Genesis.
Ideally I would like the client to keep synching from the point it successfully got to and not re sync from scratch on such errors.

I am wondering if this resync from scratch on error is necessary. I assume that corruption occurs in one (or more) of the files in the blocks or chainstate directories. If that is so then why sync from Genesis? Is it not possible to, say, delete all files post the corrupted file and sync from there? 

If there is no fundamental reason for this behaviour, I would also appreciate a pointer to the relevant code in the client.

Thanks!!
Database corruption means the state of your node is corrupted, not just data it downloaded.  Resuming after state corruption would be like resuming your life after having a lobotomy.

During synchronization your node operates at hundreds of times its normal load: Consider, Bitcoin has existed for over 3000 days yet sync will complete in a day. This tends to expose marginal hardware. "suboptimal" would not be a very accurate way to describe it, "totally broken" would be better.

USB drive in general are notorious for causing corruption. Not only are the drives themselves often unreliable, but many people often suffer issues from their USB controllers and even cables.

At the expense of several more gigabytes of space usage Bitcoin nodes could periodically back up their state so that they could recover more gracefully from corruption... but the cost of doing that is great enough that the gains aren't completely clear. Especially since hardware that is messing up is also probably likely to corrupt the backups.