Is there any way to remove edges causing a loop?
From what I can glean from here, Blender's representation of a polygonal mesh does not currently permit faces to have holes.
The rule appears to be that a face must be bounded by a single loop of vertices, with no vertex visited more than once, so..

.. a hole can be bounded by two faces, as on the left, but not by one, as on the right. That would mean doubling-back on the radial edge.
The topology of your top surface seems to be as shown in the examples below, a figure-of-eight region with two holes:

In the state you show, the region is divided into 3 faces, as at the top.
It would be possible to divide it into 2 faces, with one edge connecting the holes, as in the middle.
But not by connecting the holes twice, leaving the outer, green, face-with-a-hole.