Unity Gain Buffer as a Current Source
You have to use a power amplifier or maybe a line driver. These type of opamps can supply more current than the standard opamp. (You can find some e.g. at TI website under Amplifiers/Line drivers or Amplifiers/Power Opamps). 

If you are sure that you don't need more than 20mA, you might also find some standard opamp. Take care, that the higher the output current, the lower the output swing - check that in the datasheet.

Extra tip, because I failed to do this a few times: You also need to take care of the self-heating of the amplifiers. It is usually the sum of the intrinsic heating (quiescent current times supply voltage) and the load dependent heating (output current times the voltage drop on the opamp). Multiply this power with the thermal resistance and check whether you are in the operating junction temperature limits.

Edit: as pointed out by electrogas also check out the TI AN-31 OpAmp Circuit Collection. You find the current sink on page 21.