It seems that the data dump is almost useless, just because all the data is really old. If people want to build applications, they are limited to the basic query's available in the API. But what if you want to do more? 

Since uses for data are usually found after the fact, lets ignore that aspect for now. My question is *why* data dumps are limited to a monthly basis? Is it a technical limitation (computationally intensive, not enough bandwidth, not enough servers)? Is it an intensive manual operation for some reason? Is there a security issue? Or some other reason?

The preferred solution is a much more updated version. In the perfect world there would be a read only account with restrictions on what columns can't be viewed hooked up to the live db. Since that breaks basic sandbox rules, that's probably not possible. So why not a separate db that's synchronized with the live db on an hourly or 30 minute basis? Something that's a lot more live, better, and can be used as an alternative to the stack apps API for better statistical analysis.