What kind of circuit do I need with a 6N138 optocoupler to get proportional output?
It's not easy to do with a simple optocoupler, because the CTR is not well-controlled.
The usual solution is use a &quot;precision optocoupler&quot; like TIL300 or HCNR200. These have one transmitting LED and two matched receiving photodiodes. You use one photodiode to deliver the signal to the isolated side, and the other one to provide feedback to an op-amp circuit on the sourcing side to enable the signal amplitude to be controlled accurately.

This may look like a lot of circuit complexity, but it's likely to be simpler in the long run than calibrating a single-receiver optocoupler, compensating for temperature variation, etc.