What are the tradeoffs for more coefficients for fractional delay filter? Is there a rule of thumb for how many coefficients to choose for a fractional delay filter?  I figure there might be a trade off based on the signal frequency as well as my sample rate (this is running on an FPGA), but I can't seem to find a good rule of thumb.
More taps = more resources (multipliers and registers)
More taps = longer transient response, in some applications it can be a dealbreaker. It could cause significant pre-echo which can be problematic in some applications such as ultrasound time-of-flight diffraction.
More taps = better frequency response, your fractional delay will have a unity gain for a wider frequency range. Same for group delay.