Don't move all of the faces
Perhaps you are trying to extrude the face? This can be done by pressing E with the face selected.

To delete those added lines, there are two ways to go about doing so.

Method 1) In edit mode, press 2 to enter edge select mode. Select the unwanted edges, press X and select Dissolve Edges.

Method 2) In edit mode, press 3 to enter face select mode. Select the faces you want to join together, press F and those faces should be joined together.