Stereo to High Speed LED for Data Transfer
There is already a solution to converting stereo audio to a digital optical signal and back again that's widely used and readily available: TOSLINK. It basically is a simple conversion from the electrical S/PDIF signal to an optical format. There are many commercial chips that can convert directly between analog audio and S/PDIF (in both directions).

The only difference is that TOSLINK normally uses inexpensive plastic fiber to couple the light between the transmitter and the receiver, and you want to do a "free space" transmission. Therefore, you are going to have to focus on the optical part of the link, dealing with highly-variable signal levels and lots of potential interference.

White phosphor-based LEDs tend to have poor frequency response, mainly because of the slow decay of the phosphor's glow, although if you read datasheets carefully enough, you can find some with fast phosphors.

But you could use the slower LEDs if your receiver includes an optical filter that passes the blue light directly from the LED and ignores the yellow light from the phosphor. This might be a good idea in terms of rejecting interference anyway.