Tag storage when using HBM as an L4 cache
Disclaimer: It's been a while since I last read that book. It was an edition that pre-dated HBM. So I can't really refer to the book to find out what it's saying.

The "level" of a cache refers to how many steps removed it is from the CPU's load/store hardware. It means nothing beyond that. You can't just say "L3" and assume that it refers to the same organisation.

In a modern machine, there is usually one L1 cache per core, split between instruction and data. It may be virtually addressed or physically addressed depending on the architecture.

L2 cache is typically unified, instructions and data. But is there one per core, or one per die?

And what does L3 cache mean? Is that the one shared between cores on a die? Is the GPU on the die, and the L3 is shared between the CPU cores and the GPU? Is the L3 cache off-chip? Or perhaps the tags are on-chip and the data is off-chip? Maybe this is a NUMA machine, and the L3 is shared between all processors on the same board?

I think the issue here may be that any signal that has to go off the chip is more expensive than one that stays entirely on the chip. It usually requires higher electrical currents, and more time since the tolerances of a circuit board aren't as tight as they are on a die. So if you can't fit an entire cache on a single chip or package, it can make sense to split the tags from the data.

If you are a CPU die, for example, and a different die invalidates a cache line (e.g. it changes state from Shared to Exclusive in the MESI model), then all levels of your cache need to be potentially invalidated. That is an operation that only requires accessing and modifying the tags, not the data. So keeping the tags on-chip and storing the data off-chip could make sense here. This was extremely common back when you couldn't pack quite so many transistors on a die.

I don't know if any of that helped.