How to remove a part of edge loop which deforms my object? I have just started off with blender, so please forgive me if I misuse the terminologies.

I have created an object by merging together a cylinder and a cube as shown in the screenshots below.
I created edge loops to sharpen the corners, which worked fine. It caused some deformities at the top face of the cylinder which I fixed by creating an inset. 



But on the sides of the cylinder, it creates a pinched effect.

I tried for really long to remove the extra edge, but could not do so without introducing more complexities and deformations.

Is there a way to deal with this?
It seems that this must be a very common scenario.
Select the edge loop by pressing Ctrl+Shift+RMB
Then pressX or Delete button, right on the bottom you will find an Edge Loop option. That should do the trick. 

Edit : Make sure you are on edge select mode.