Can I switch sound like this?
Yes, that should work.  Basically your question seems to be whether you can switch a audio input, and the answer is yes.  There may be a pop at the switchover, but it doesn't sound like that is a issue.

I'll assume the DPDT switch is for switching both channels of a stereo pair.  If so, that's fine.  I just want to point out that since everything is in close proximity presumably operating with the same ground, the grounds don't need to be switched.  Each channel requires only a SPDT switch to select one of two inputs.