Why is the speedbrakes rate of retraction 25 seconds when used at speeds greater than 315 knots/.75 Mach on the A320?
Please read the entire answer to understand the matter even if the answer goes a bit beyond the question.

On the A320 wings you have panels that may open upward and that could be used to fulfill 3 different functions: ground spoilers, speed brakes, roll spoilers.

The outer panels are used in addition to the ailerons for roll function, that is helping the ailerons only on one of the wing according to the turn direction.

The central panels are used in flight for the speed brake function as follows: when the pilot order is beyond a limit the Fly By Wire will provide protections because an excessive drag might damage the wing structure, so the higher the speed the lower will be their maximum angle of deployment and their speed of deployment even if the pilot gives a full order of deployment.

The drag produced by the air brakes causes a mechanical torque on the wings roots, a sudden removal of the drag would be similar to a stress cycle, therefore deployment and retraction are managed to protect the wing structure.  Any pitching effect of these surfaces in flight is automatically compensated by the FBW through the elevators.

At landing the ground spoilers function is as follows; all the surfaces are deployed fully if the function is armed.