Remove just the Location
Make sure the object is selected and open a Dope Sheet window.


Each keyframe is represented by a diamond (white = Unselected, orange = Selected). You can use the usual tools to change the selection - eg, right-click to select a single keyframe, A for Select/Unselect All, Box Select (B), Shift for multiple select, etc. To Select all keyframes on a channel you can select one of the keyframes and then press L to select all linked keyframes on the same channel. 

Select the keyframes you want to delete and press X to remove them.