Low loss current sink
Split R3 into 2 parts.

Place a TL431 (2.5V) or TLV431 (1.25V)  or cheaper clamp regulator if you can find one (note 1) from centre point of split R3 to ground.

Voltage drive is now held constant by clamp regulator.
 When drive is low regulator is off and no drive current.
 When drive is high clamp sets upper limit.
 Response time of clamp may set an upper limit on PWM frequency - but so will op amp. Getting "fairly iffy" by 100 kHz. Possible with proper attention to datasheet. 


Note 1:  zener diode can be used as the clamp with some loss of accuracy. Even if N of these are driven in parallel they are  liable to be driven from  the same sort of input so the zener will behave similarly in all cases.


For a solution between zener and clamp regulator in cost for N circuits have a single clamp regulator and connect midpoint of split R3 via a diode to clamp regulator. Liable to produce an entirely acceptable result. As long as clamp is powered by a separate resistor to give it holdup current the clamp is not turned off when PWM is off, which is an advantage. 

Digikey have TLV431 at $US0.54/1, $0.484/10, $0.32/100, $0.186/1000, $0.142/10000 

About $US0.07 in China or Hong Kong  in 3000? up quantity LR432 - Leshan Radio - genuine Motorola based part (joint partnership). 



Variations between channels when using a common clamp:

Variation liable to be acceptable if light output is used for "normal" human vision applications. 

I'd imagine you could clamp each at within 0.1V of each other and probably 0.05V. Very easily tested. If the clamp is stiff with load, which it can be within very tight limits, then the variation is in diode Vf with current and between diodes. Current will vary slightly with drive voltage and value of driving resistor but voltage is liable to vary only say 10% max (0.3V in 3V3 say) and resistor can be 1%. Diode drop is theoretically a function only of silicon properties and temperature and current. 

If you used 0.5% TLV431 or equivalent at 1.25V the clamp would be about a Vbe (= diode Vf) higher at about 1.9v say. Diode Vf varies with current and temperature as above. 

Varying 1.9V by +/- 0.1V ~= +/- 5% and probably actual of 5% total.
 With Rset, R2, R3 = 1% you still get a % or few error from them so clamp error is not vastly  more.  

Eye/brain can see 


50% difference between two lights non-simultaneously viewed 
or say 25% side by side illuminating a general scene or
A few %  - maybe 5% - 10% range, with lights "wall washing side by side. 


If you were using this for eg mixing RGB+? colours or similar you may care more. But I think that hue variation distinction is liable to be fairly insensitive to most people. I have carried out light level tests as above but not hue variations - but my general impressions on hue come after looking at a vast number of LEDs "along the way".