Animating a growing low poly tree
So I'll just post this as an answer but if anybody has a better suggestion feel free to share!
So I managed to get it to work thanks to moonboots's comment

As suggested I made an armature and just scaled down each branch seperatley and then rotated it so it's hidden inside the mesh. It looks fine from a distance but not so great up close