Why is the memory of common GPUs so constrained?
Basically, the reason is because the memory of the GPU is much more expensive. This is due to the fact that GPUs using a very high memory bandwidth. A normal GTX 480 has 384 bit wide bus, which allows a maximum transfer rate of ~180 GB/sec. However, an Intel i7 CPU using a 192 bit wide bus  has a maximum of about ~30 GB/sec depending of what type of RAM you use. This means that the transfer of data in the GPU is about 5x faster compared to a CPU. Therefore, memory for GPUs is much more expensive but you can get more transfer ratio.

Additionally increasing more chips, the subsequent increase in capacity could end up messing with the timing of transfer of data.