Suggestions needed to replace relay on this circuit
I don't think you need solid state relays and such; they are terribly expensive, at least $35 to get one with a low on-state resistance, and the best turn on/turn off times are in the order of 50-100 µs, not all that better than what you are using now.  Instead, you should be able to just use a N-channel FET as a low-side switch.  The IPB009N03L has a very low R\$_{DS(on)max}\$ of 0.95 mΩ, a turn-on delay + rise time of 50 ns, and a turn off delay + fall time of 125 ns.  It is designed to work with a logic level input since it has a minimum V\$_{GS}\$ of 2.2V.  It is available at Digi-Key for $2.87.

The main disadvantage of a low-side switch is the switched circuit is no longer connected directly to ground. However with such a low R\$_{DS(on)max}\$, and currents under 20 mA, it shouldn't matter.

But if you must use a high-side switch, you could use the P-channel IPB180P04P4L-02.  It also has a low R\$_{DS(on)max}\$ of 2.4 mΩ, a turn-on delay + rise time of 50 ns, and a turn off delay + fall time of 265 ns.  It has a minimum V\$_{GS}\$ of -2.2V.  It is available at Digi-Key for $2.20.