If banks started simply depositing money with each other, then what else?
The purpose of paying interest is to get people to lend you their money (by depositing it) so you can lend it out at a higher rate of interest. The argument here is that if the deposits outstrip the bank's ability to invest/lend, they don't need to offer as much interest to get those deposits. However, given current banking practice i don't think there is ever going to be more deposited funds than the bank can find profitable uses for, so the thought experiment sorta falls apart.