GPU vs CPU on chip memory
It seems that the latest NVDA GPU chips have >10GB of on-chip memory.


On-package != on-chip. The HBM are stacks of several chips right next to the main GPU. Consumer grade GPUs have the memory soldered to the PCB in normal BGA packages.

GPU memory works "better" because there are no connectors that could impact the signal path between chips, allowing higher frequencies.

PC main memory uses sockets, this allows you to upgrade the RAM by changing or adding modules. But these also degrade the electrical signals, and thus performance.

Also note that only a few MB on the GPU chips itself are SRAM (caches) - the overwhelming majority of memory is DRAM.