How to prevent inside faces and non-looping edges when mirroring?
The inside faces are probably due to the vertical edge at the far corner of your base model. Deleting that edge (and associated faces) should help. You don't need that geometry in your model, anyway.

For the surface vertices, did you try "remove doubles" with a sufficient margin to collapse the unwanted doubles, while keeping all others. It helps if your edge vertices are exactly on the XZ or YZ plane.