Could abrupt flight controls input cause damage in these circumstances?
Now you know what a terrible idea it is to do "fool around" stuff with a passenger.  

The AA-5's maneuvering speed is 122 MPH so you can't overstress it at 105; it will stall first (and you'd have likely snap rolled it) so you should be fine.

When in doubt, the rule of thumb is, when it's metal, if it's not permanently bent, it's good to go.  The metal structure is a big spring (elastic deformation under load and returns to its original shape below the "yield" threshold) and if you don't go past its elastic limit into the plastic range (the yield limit - permanent set to a new shape after bending) it just springs back and Bob's your uncle.  A single application of a near overload to a metal structure has no significant effect fatigue wise if you stayed in the elastic range. 

If you think you might have over stressed it past the metal's yield point so there is a permanent set, there will be telltale wrinkles in the skins of the wings or tail that weren't there before. If you know what to look for, I wouldn't call for a mechanic, but if not sure what you're looking at, you should have an expert take a look.

But, like I said, a pull or push on the AA-5 at 105 mph can't overstress the airframe in the first place, so if you were really going that speed, I wouldn't worry about it.  I would just resolve to be more careful about passenger briefings and "horseplay".