2 sources one speaker - for old rotary phone
It sounds like you don't need to actually mix both audio sources together, just switch between them. Therefore you can get by with a dual SPDT analog switch, like the TS5A23159.



I chose a dual SPDT switch so you wouldn't have to deal with any issues tying the grounds of the two sources together.

The device is available in a DIP from Digi-Key for under a dollar.

Note: the supply voltage can range from 1.65v to 5.5v, which makes it ideal to work with 3.3v devices such as the Raspberry Pi.  However the lines being switched cannot be greater than the supply voltage + 0.5v, or more negative than -0.5v.  If your audio lines exceed these limits, then consider my relay suggestion below.

========================================================

Another option would be to use a reed relay, such as the HE700; you would need two of these since they are only SPDT.



They are available from Digi-Key for $3.87.  I have used several of these myself and they work well.

The downsize here is that since the relay requires 5v, and the Raspberry Pi can only output 3.3v (and its pins are not 5v tolerant), you will need a transistor circuit, either a BJT or MOSFET to drive the relay.  There are plenty of answers here about driving relays from a microcontroller so I won't repeat that here.