What are these metal tags attached to my poured concrete basement wall?
Yes you can remove them. They are not a complete unit anymore. There was a wire extending from the center of the square disk, you will still see the nib that held the wire. The purpose of the wire was to push one type of insulation or another over it. The insulation was held in place by another disk that grabbed the wire and it would not allow the insulation to be pulled back off, at least not without some force to pull it through the disks. 

Nowadays there are adhesives that will do the holding, and in some places, this type of retaining system is still used. For fiberglass batts or rolls is what would come to mind for using these.