How to setup an I2C bus which stays valid if a slave is powered down or fails?
The 3 Arduinos are currently being powered by independent USB power supplies, but I intend to power all of them from one single power supply using their Vin pins.


I think your problem here is that if you power down one Arduino, it isn't really powered down, because it will get "parasitic power" from the I2C line, particularly from the pull-up resistors which should be there. The protection diodes (internal to the processor) will allow power to flow from the A4 and A5 pins to the Vcc pin of the chip, which will then power up, probably in some sort of indeterminate state.

Since you are planning to power them from the same supply anyway, your problem should simply go away when you do that. Connecting a device with active inputs to a circuit (ie. a voltage present) when the device itself is not powered up, is not recommended. Exceptions would be devices that are specifically designed to handle that, for example, commercially produced printers via their USB input.