Differential to single ended audio signal
You can use variable gain amplifier like this one, that you can connect either to a potentiometer, or to a DAC, it will come much cheaper than a digipot and solve lots of the issues related to it to the design.

There are plenty of those, controlable from an analog source, or digital like I2C, SPI...