Advice for first time TA in mathematics
Make sure you are thoroughly on top of the material before every student interaction.

This sounds obvious.  Moreover, it sounds like it could be condescending: if you're, say, a PhD student and you're teaching, say, freshman calculus, haven't you known the material cold for most or all of your adult life?  Probably yes...in some sense.  But the sense in which you need to know the material cold in order to answer students' questions effectively is much stronger.  

Imagine that when you were a calculus student you were assigned to do ten applied optimization problems.  Let's say that you knock out eight of them immediately, spending no more than 5-10 minutes on each.  There's a ninth one that you keep doing the algebra wrong and it takes you 20 minutes instead, but no problem: you get it.  There's a tenth one which has some weird thing inscribed in some other weird thing and the first time you think about it you just get a bit confused.  No problem: you put it down for a while, go about your business, and you come back to it later when it's the one thing occupying your attention and you solve it correctly.  How good at applied optimization are you?  The answer is you're excellent: an A student with room to spare.  This level of skill more than qualifies you to be a TA for this material...but you had better do some advance preparation before the session itself.  As a TA, the students are very likely to ask you about the two hardest problems.  If you make more than one or two algebra mistakes in front of a large group, it's not great for them or for you.  If you try to solve a problem and fail, then -- even though you know you'll get it later, and even though "getting it later" is absolutely what matters in the world of actual mathematical work!! -- then the session itself is probably a failure, the course instructor is going to get some "How am I supposed to solve this problem when even the TA couldn't..." questions, and she is perhaps going to be less than thrilled with you.  When I was a TA it seemed to me that my on-the-spot problem solving skills actually had to be at a higher level than if I were actually teaching the course: if you are teaching the course, then you are controlling the pace of the information flow and choosing the problems. (In particular, if one of 50 problems from some section of a textbook is slightly confusing to you, you'll probably just not pick that problem!)  I haven't had to deal with course TA's for a while -- in my current job, graduate students do grading but rarely TAing -- but upon reflection I still find some truth to that.  

(By the way, I have picked what was and maybe still is for me one of the shakiest topics from freshman calculus.  Of course there are lot of calculus problems that I really have always known cold...and after many years of TAing and teaching I really know which is which.  But there is something in the psychology of math students which makes you think that you should know everything cold or there's something wrong with you, so why don't you proceed under the assumption that there's nothing wrong with you and see how that works out.  What I'm saying is that there is likely to be something that you will screw up if you wing it...and that doesn't make you a bad math student, it just means you shouldn't wing it.)

How do you make sure you are thoroughly on top of the material?  As a first time TA, I would actually work every problem in advance.  Maybe this sounds obvious or most TA's know to do this...but I don't think everyone does.  Even as an experienced instructor -- say, at 10 years past my PhD -- I have had the experience of teaching a new course, thinking I knew exactly how something was going to work out, but because I hadn't written it down in as much detail as I was going to cover it in class, encountering unexpected challenges -- more often expository, but sometimes actually mathematical -- when it came to the lecture.  In fact this happened to me about two years ago when I taught a first course in linear algebra.  I really know linear algebra: like most mathematicians, it comes up in my work, and unlike many mathematicians I have a published paper in the area.  But there are certain things that I didn't have to worry about as a research mathematician and did have to worry about as a linear algebra lecturer.  I was actually rather embarrassed at the shakiness of my lectures.  I compensated for this by being more attentive to the students' concerns and writing more straightforward exams ("How are they supposed to solve problems when even I couldn't..."), and in the end I was surprised that the teaching evaluations were as satisfactory as they were: not great, but no worse than certain calculus classes where my lectures were pulled off exactly as I wanted to but the students were not as thrilled.  

By the way, I well know that a lot of people will be tempted to roll their eyes a bit at an answer which is entirely focused on the content of the subject and says nothing about pedagogy, is focused on the mentality of the TA rather than that of the students, and so forth.  But this is where you have to start.  A first-time TA who knows how to solve all the problems and is just starting to learn about everything else is doing okay...and in fact better than some.  The next most important thing is learning to present mathematics in a clear, orderly way and the best and onliest way to do that is to get a lot of practice doing so.  The next time you teach the same course is when you really get to concentrate on the "higher aspects" of teaching. 

Good luck, and good luck to your students.