Are low ESR caps away from the LM317 liable to cause instability?
The datasheet states the output cap (C3) is not needed for stability,
so I will omit it.

If you can live with the poor transient response when the load changes and the inherent noise produced by the LM317 then that's fine.

I have also read low ESR caps for C3 can cause instability

Output capacitors can cause instability. After all, the regulator is a control-loop and it has a phase margin of maybe 60 degrees. Adding the wrong amount of capacitance can cause a control-loop to become unstable. Too little can cause a noisy output and too much can lead to sluggishness in response when starting. Less than enough but more than too-little can cause some regulators problems sometimes and, this is alleviated (in some designs) by having (circa) a couple of ohms in series with the capacitor (aka ESR).
On the other hand, the LM317 is pretty stable (albeit being a very noisy regulator): -
CO improves transient response, but is not needed for stability

Stick to the recommendations: -