Editing vertices in multiple planes
If I make W > Remove Doubles it does remove doubles. So you have a lot of vertices overlaying. Also, there are a lot of edges crossing faces but that are actually not part of these faces, that's why when you drag some vertices some edges stay still. I guess when you modeled your mesh you used F to fill the gap between two vertices, but as they were already part of a face, instead of cutting through the face, it has created an independent edge over the existing face, you should have used J to join these vertices. Or maybe you had a vertices grid and you pressed F to fill and it created faces over other faces? Anyway, the best way to fix all this mess is to delete all your faces and keep only vertices, remove doubles, and correctly fill the faces.