I have doubles, and remove doubles won't delete them
Looks like you don't have duplicate vertexes. Looks like you have an edge on the inner vertexes and another edge from the farthest vertexes. The edges overlap where you circled.
You can try to delete the edges and create them correctly, then recreate the needed faces.