Why does the die of a desktop CPU need to be so small?
Cost is proportional to die size, so for that reason the die is usually optimized to be as small as possible to meet the performance requirements.  

The number of die you can get per wafer is extremely important in a high cost process like those used for desktop computers.

There are other reasons as well, like routing very high speed signals, but cost is the main driver.