audio-taper potentiometer in 50-100 ohm range
This might be an option: -



It can be a very accurate constant current source and relies on the fact that the op-amp takes an input voltage referenced to the positive supply voltage rail (Vin) and, by op-amp action, supplies enough base current to the transistor to ensure that that voltage also appears across R1.

This means that the current through R1 is Vin/R1 and that the current through your load is also the same current (plus a little bit of base current). I normally expect the current output to be accurate to within 0.5% of the demand voltage (Vin) and the resistor value R1. So, if you need accuracy make R1 a 0.1% resistor.

There is also a version of this that can use a P channel MOSFET. Here's one that is dual stages to give you the control voltage referenced to the 0V rail: -



An LM317 is OK as a current source but it isn't very accurate and will tend to drift with temperature. It also needs a couple of volts headroom between input and output voltage to do the job properly. There is also a minimum current that you cannot get below due to the device needing a couple of mA to actually work (this flows into the load). Also, as a dynamic current source it's impedance is not very good at high frequencies.