How to move vertices without duplicating them?
This may be because you have a couple duplicate vertices in the same location.

Select all your vertices using A in edit mode and then press W to bring a dropdown menu. Either select "Remove Doubles" or press R as it's faster. Now try moving your vertex. It should work.