Stray vertex that refuses to be deleted
That doesn't look like a vertex, it looks like the Object Origin - the point which marks the location of the object in 3D space.

You should be able to confirm this by looking at the status bar at the bottom of the window. It should show 'Verts 0/0', showing that all vertices have in fact been deleted:



You can't get rid of the object origin without deleting the object, which you can do by switching to object mode and pressing Delete.