After a deformation operation on polygons, how can I check for and fix inverted polys? I'm doing a quick and dirty automated deformation routine on a polygon body.  If it was a tree, and my axis was in the center of the tree, I would like to bend the body by bending the axis.  I would use a simple influencing algorithm to determine vertex point xyz displacement.  

On the concave side of the bend, how can I check for polygons that have overlapped or passed through each other, and how can I fix such polygons?

Thank you!
If you have an oriented triangle mesh (a non oriented surface would be the Moebius Strip for instance), you can check the triangle inversion by looking at the signed area of triangles. The signed area is positive when the vertices are in CCW order and negative if they are in CW order. If your triangle vertices are all sorted in CCW then all triangles with negative area are inverted. Fixing that depends on the deformation technique, I don't know your case.

Checking self-intersection is more involved. You should actually do Collision Detection of the mesh with itself. Collision detection is fast but need to implement proper data structures such as AABB Tree for narrow the amount of triangle-triangle intersection tests. Collision Detection is usually performed using very low-poly version of the meshes. In some computer games, the bounding volume hierarchy (spheres, AABBs, etc) that approximate the mesh is the only thing considered for collision, no the mesh itself.

Once a collision is detected, there are several ways to response, applying a  bouncing force for instance, is common.