Identity covariance matrix, decorrelated data?
Having only taken a few courses on this stuff, I'm going to offer my understanding on this.

The main reason seems to be simplifying, as you assume. It manifests in different areas though. One is mathematical - if your co-variance matrix is identity matrix then the math becomes much easier. The one directly following from this is computational - inverting a matrix is very costly, but identity matrix is its own inverse, so that's perfect.

Another part of the simplicity is definitely the ease of interpretation. Independent variables are easy to explain to a non-technical person, but as soon as you get into covariances, that goes out the window.