Does this rdists package provide actual (predicted) response time for each observation?
Not very familiar with this package, but per manual (bold and italics added):


  ddiffusion gives the density, pdiffusion gives the distribution function, qdiffusion gives the quantile function (i.e., predicted RTs), and rdiffusion generates random response times and decisions (returning a data.frame with columns rt (numeric) and response (factor)).


So, it looks like if you want predicted reaction times, you should use the qdiffusion function (but note that it is a quantile function).