Is there a way to speed up the way Mac responds to my trackpad clicks?
Do you have trackpad dragging enabled?  If so, it has to wait a moment after you tap in case you tap again and turn the action into a drag -- until then, it can't tell if you want to just click, or click-hold-drag.  The only way to eliminate this delay is to turn off trackpad dragging, so there's no ambiguity about what it should do when you tap.