Isolating batteries from charger
A relay would really be the only way to completely isolate the the charging battery subsystem from the consuming battery system. Using solid state components to switch the battery connections would probably be good enough, even with a shared ground, for most applications and users. But technically this still leaves a connection. On the other hand for some audio purists, and it seems that you fall into that category because you are even talking about using batteries to power your audio system, you would have to use a multi-pole relay to connect both the + and - sides of the battery to one system or the other. 

Such system could very likely provide your intention of "clean battery power" to the audio system. There are some short comings though -


During the switch over from one battery to the other there would be an annoying loss of operational mode of the audio subsystem for a moment. 
Some audio equipment may give loud pops when the power source is suddenly connected or disconnected.
If the charger is not capable of charging one battery faster than than the consumption rate off the other battery then you maximum listening time, with one annoying battery switch over event, would be the time afforded by two charged batteries.