How critical is TA experience in grad school if I want to pursue a career in academia?
Two years is probably enough, especially since your two years' of TAing went so well.  You may want to concentrate on demonstrating yourself as a researcher now.  However, if you have time to spare, I'm sure you could offer your services to the person in your department who coordinates TAs.  As I probably don't need to tell you, you'd want to keep your advisor in the loop.

Consider that a fellow or research assistant is sometimes pressed into service to lecture in place of the advisor in case s/he has to miss one or more classes.

When you feel ready, you could expand on that idea, and ask someone (probably your advisor, but it could be someone else too) if you could be a guest lecturer for a particular topic in a particular class.