What would happen if flaps are extended beyond 40 degrees?
Depends on the kinda of flaps installed on the aircraft; in the example above with fowler flaps, the design would begin to just create excessive drag without gainful lift and would require a pretty bulky structure to accommodate for the aerodynamic loads imposed upon them.

Now some other types of flaps like the split flaps or dive brakes like those found on the Spitfire fighter were designed to extend 90°.