What does this icon mean ? and How can I erase it at once?
Let's go one by one:

The 'icon' you're reffering to is a so called empty.
The ones in your file contain no geometry and are generally useless.
You can delete or hide them.

1) How to delete all these empty objects.
Press 'B' on your keyboard for box select. And then right click -> delete.

Be careful not to delete real geometry when cleaning up your file.



2) You can get rid of those black lines here:
Overlays->Relationship Lines and Overlays->Extras