What limits a mobile phone to be upgraded to new Android version? What is the main reason that you can't upgrade your phone to new Android version?

Is it the lack of drivers, or simply hardware limitation? Are there other aspects?
Somebody needs to write and test the drivers and core apps. That's all.

Most applications that are written for Dalvik (i.e. most apk programs) are pretty portable across different Android versions, but not so much for core programs that were written in C, the Linux kernels and drivers, and the Dalvik VM itself. At the very least, these programs need to be recompiled; occasionally, the new Android version may require new capabilities from the drivers, and someone will need to write the code for them.

Additionally, for official releases, manufacturers and carriers often adds value-added customizations; these customizations need to be ported, tested, and often debugged against the new framework. In deciding whether to write upgrades, manufacturers (and to certain extent, carriers) faces a dilemma, they make better profit by selling to you a new device instead of writing upgrades for free, but poor upgrade history will put off customers as well; so they will often drop support for older phones and only write one or two version upgrades at most.

Hardware limitation has never been a problem for upgrades. While it's certainly true that certain new features might have a much higher hardware demand than older phones can conveniently handle (e.g. Live Wallpapers), these features are usually either disabled or left to give sub-optimal experience.

Other limitation is space limitation. Some phones have very small internal memory, and Android tends to get larger for every upgrade. To port newer versions for these phones, some sacrifices has to be made; whole libraries may be left out (e.g. who needs the NFC on a G1?). These modifications needs to be written and tested.