What kind of delay does the A320's fly-by-wire system add?
Yes there will be a delay, but the delay caused by the control loop is really tiny. I've seen position control loops run successfully and stable at a couple of hundred Hz for simulator motion systems, and the time delay is just one iteration frame = less than 10 msec.

And as @ymb1 correctly points out, if we deflect any control surface the end position is subject to actuation forces, aerodynamic pushback forces, inertia of the surface, cable stretch and -friction, aero damping, maximum flow velocity through the servo valve etc etc. The total transfer function of stick input => surface position is such that an extra time delay of 10 msec in the total loop is not really noticeable. And control position is only the input of the aircraft response, with aircraft inertia playing a large role.

So my answer would be: a tiny time delay that can be totally disregarded.