Slingshot Dynamics Cloth + Hook
I figured it out myself. I did the first pulling back of the slingshot like I did before. From there I duplicated the mesh and applied the cloth simulation as a Shape Key.

So I now have the current mesh as a shape key where the first mesh stops with its cloth simulation. Then I had to change the order of the shape keys so on top I have the output mesh from the cloth simulation and on the bottom a shape key of the very initial mesh (The idle shape of the slingshot). So on the second mesh in the cloth simulation I just put the initial shape as the rest shape. So when I play the animation it first plays the first cloth simulation and when I want to shoot, I play the second cloth simulation with the so it snaps back to the rest shape defined in the cloth simulation settings.


How it looks now: