1

I've been parsing the blk*.dat files using fread().

Let's say this is the basic structure of a blk.dat file:

[magicbyte] [blockheader] [transactions]
[magicbyte] [blockheader] [transactions]
...
[magicbyte] [blockheader] [transactions]

When parsing, I start by reading the first 4 bytes.

  • If it's equal to D9B4BEF9 (magicbyte), I gather the [blockheader] and [transactions] data.
  • If I get NULL, I assume that I've reached the end of the blk.dat file and move on to the next one (if there is one).
  • If I get 00000000, I assume that the blk.dat file is incomplete, and I keep re-reading it.

Is this a reliable way to parse the blk.dat files? The 00000000 check is what I'm most unsure of.

inersha
  • 2,928
  • 1
  • 17
  • 41

1 Answers1

2

It is perfectly possible that *.blk files contains gaps of zeroes, or even partially written blocks. In modern versions (since 0.10 or so) this won't happen during normal operations, but it can't be prevented in case of abnormal shutdown (crash, out of disk space, ...).

If you want to be sure, you need to scan the entire file for potential blocks (they'll always start with a magic + length descriptor, so you can quickly skip many things). That, or use the getblock RPC instead, which uses Bitcoin Core's prebuilt index.

Pieter Wuille
  • 98,249
  • 9
  • 183
  • 287
  • Thanks again Pieter. I can understand parially written blocks, but what would be the reason for the presence of gaps or zeroes in a *.blk file? – inersha Nov 13 '16 at 19:39