pressing on a vertices displays this
You have enabled a gizmo.

To change or remove it, make the tool bar appear using T, if not already displayed. Then choose another one or choose the selection box tool.