How do hierarchical deterministic wallets work transaction-wise? As far as I know (and as explained here) hierarchical deterministic wallets keep a master key pair (private and public). When using them the public key is newly generated in every transaction. I understood that child keys are generated by elliptic multiplication of parent keys. Benefits associated with HD wallets are claimed to be:


Better privacy because new public keys are generated in each
transaction;
It is only needed to store one master private key, as
all the other keys can be derived from it; and
It is possible to
store the master public key on an insecure server and still generate
as many addresses as needed without having to leave the private keys
on that server.


While I do more or less understand the theoretical concept, I struggle with the practical implementation of the process, and in particular with how does the system associate the balance from several transactions (allocated to different derived public keys) to the same bitcoin "account" or master public key?

Anyone able to make a "for dummies" clarification?
how does the system associate the balance from several transactions (allocated to different derived public keys) to the same bitcoin &quot;account&quot; or master public key?

The system must known the XPUB (often the master public key) for it to generate all of the addresses to 'watch'.
While we're on the subject of master public keys and privacy, there are many things that you can do wrong. Sharing your master public key (XPUB) is a bad idea, as the other party will be able to see all of your addresses. I've seen a lot of projects doing this, destroying your privacy basically.
BIP47, still a draft and not implemented, aims to solve this privacy issue and still offers the infinity addresses benefit of HD wallets.
But what you describe, the act of sharing a master public key and letting the CLIENT derive the addresses is bad and isn't the case in most systems. It's a lot more likely that the SERVER derives the addresses and keeps the master public key secret.
So, let's assume the following:

We have the public key through which all other new public keys (and thus new addresses) can be derived.

We have received one transaction on address 15.

We're bootstrapping a new node and doing an initial block download, and thus have processed 0 blocks.

We lost our previous wallet.dat file, so we don't have a single clue on which addresses we've received any money. We recover our wallet through the public key as described in 1.
At this moment the HD wallet doesn't really know on which addresses it has received any money, but to check if transactions belong to us it has regenerated a &quot;LOOKAHEAD&quot; of the first 20 addresses. Those 20 potentially used addresses get added to the keypool, all new transactions to these addresses belong to us.


A block gets processed with a transaction to address 15, this falls within our lookahead of 20 so we're able to pick it up. We regenerate more lookahead addresses once we've found a transaction to one of our addresses.
This lookahead is important, let's assume we took a lookahead of 10, we wouldn't be watching on address 15, only up to address 10. We wouldn't see our transaction. (note: this only applies if you've lost your wallet.dat, and recovered from seed)
BIP44 specifies the &quot;address gap limit&quot; to be 20.

Address gap limit is currently set to 20. If the software hits 20 unused addresses in a row, it expects there are no used addresses beyond this point and stops searching the address chain.