How to remove marker from openlayers map
Finally I managed to deal with it. In case someone else needs the answer:


you choose the selection tool (the one with the cross)
click on a point and press DEL