What is the difference between two universities if students from either can take classes at the other?
Three key things to keep in mind when considering these types of arrangement are:


Different universities have different requirements, and taking courses from the one university will often not clearly satisfy requirements from the other.
Just because students can take classes at another university doesn't mean they will find it practical to cross town to do so.
Certain courses are restricted even within a university (e.g., Harvard's legal and medical programs).


Thus, for example, my observation of the Harvard/MIT arrangement that you provide as an example is that few students take much advantage of it.  Early undergraduate is dominated by core curriculum requirements that are incompatible between the two schools.  Mid-undergraduate likely focuses on requirements from one's major which are likewise either incompatible or duplicated.  It is only in upper-level electives, where the class is strongly individual to a professor, that there is likely to be a strong benefit for taking courses at the other university.  Even there, however, with two strong institutions there are a lot of good and interesting courses to be had "close to home," and students rarely choose to cross town unless they are strongly motivated towards a particular course.