Can you have two sources run to one speaker if the power is cut to one?
Can I have both devices soldered to the + - lines of the speaker


Not in the general case, no.  


  or will this cause problems?


Quite possibly, yes.


Amplifier outputs aren't really designed to be backdriven by something else. Ideally they are somewhat robust against electrical faults (for example shorts) but not always.  Before IC amps with build in protection circuits, blown fuses or transistors could happen.   Note that two amplifiers trying to drive opposite voltages are similar to a shorted speaker wire.
The specifications of many active semiconductor devices specifically prohibit having voltages on pins when the device itself is unpowered.
Even if there is no damage, the unused source may load the speaker lines, taking power and quite possibly causing distortion.


A "mixer" would only be a solution for low level outputs which you then fed into a power amplifier - you don't really mix power amplifier outputs.

A switch (possibly even more contacts on your relays) could be an option.  Running an amplifier without a load isn't a good idea either, but many will tolerate this briefly even if they don't have protection against other kinds of faults.