Should I correct mistakes in student presentations?
Mistakes should be corrected, but the how can be finessed. You are already correct in giving the private feedback to each student, but you can add an additional level to the game. 

Instead of you correcting the student in the live session or later, part of each student's total work product is to produce a report on their own that is distributed to the class soon after the presentation. Much of this can be prepared prior to the presentation and only updated after your private feedback. In it, you can require that the student correct any known defects in the presentation explicitly. They can admit to having made a mistake and provide the correct information without any confession of guilt or error. The info was wrong - here is the real skinny. 

There are a lot of reasons for misstatements in a public presentation. You and I make them as well, and we correct them as we can. With students (and new teachers) some of this is actually just a result of stress. So the correction shouldn't increase the stress or give the student any reason to feel bad about themselves. The correction comes from them, even though it was information from you, perhaps, that set them straight. 

To simply let it go, misleads the other students of course, so that is a poor solution. But lowering self esteem is also a poor solution. Make the experience a positive one, even though the student is cleaning up her/his own mess. 

The written report has an additional benefit, of course, as students need to practice technical writing in any case.