How to handle incomplete blocks when reading blk*.dat files? I'm trying to write a parser for blk*.dat files from scratch. Right now, I can read blocks, extract the header fields and calculate the correct block hash based on the header fields. 

I just tried to run the parser through all blk-files and it failed after successfully reading the first ~300 blk-files. When inspecting the failing file with a hex-editor, I found two "magic byte fields", only separated by 4 other bytes. 

 

Since the mandatory block-header is supposed to be 80 bytes in length, I'm now quite confused.

I found three probably related questions:


blk file error when reading - did something change in the format?
How can you tell if you&#39;re at the end of an incomplete blk*.dat file?
Zeros in blk00*.dat files


My guess, based on these other questions, is the following:


This is an 'incomplete block'
This happens when a block can only be partially downloaded, or written to disk.
Deleting the affected blk files and re-downloading parts of the blockchain doesn't make sense, since this can be expected to happen again. 
The parser should be able to skip these blocks.
The overall blk file is not missing actual data and is not corrupt.
The 'incomplete block' will be downloaded and written again further down in the blk file (or to an other blk-file). 


Which of my assumptions are correct? .. which are not? Am I missing something?
I'm going to answer my own question. These are my conclusions based on the comments from Nade Eldrege and eponymous.

Assumption 1: This is an 'incomplete block'

Yes, this is an incomplete block. More specifically, it is not part of the blk file format.

Assumption 2: This happens when a block can only be partially downloaded, or written to disk.

Yes, evidence strongly suggests that this happens when the bitcoind is interrupted / killed while writing to disk. If restarted, it just continues to download and write new blocks to the blk file. It doesn't clean up incomplete block. 

Assumption 3: Deleting the affected blk files and re-downloading parts of the blockchain doesn't make sense, since this can be expected to happen again.

Yes and no. Yes, because deleting and re-downloading the blk files without interruptions solves the issue. When writing a blk-parser, however, it should be able to handle these incomplete blocks. For example, by skipping to the next valid block.

Assumption 4: The parser should be able to skip these blocks.

Yes. See #3

Assumption 5: The overall blk file is not missing actual data and is not corrupt.

I'm not sure, but since the bitcoind just continues with downloading blocks and works without any issues, one can assume that the incomplete block is just  re-downloaded afterwards.

Assumption 6: The 'incomplete block' will be downloaded and written again further down in the blk file (or to an other blk-file).

It seems so. See #5