How to remove a part of edge loop which deforms my object?
Select the edge loop by pressing Ctrl+Shift+RMB
Then pressX or Delete button, right on the bottom you will find an Edge Loop option. That should do the trick. 

Edit : Make sure you are on edge select mode.