How to apply rigid foam insulation to foundation with epoxy plugs?
I'm not familiar with epoxy repairs, but I'd guess that those holes were drilled to allow material injection and the plugs merely retained epoxy while it cured.


  Can I remove the injection ports after the completed repair?
  Yes, but first wait 24 hours to allow the LCR injection resin to harden and fully cure. A hammer and chisel will easily remove the injection ports, and the crack sealer can be sanded, ground off or painted.
  
  https://www.polygem.com/lcr-faqs-frequently-asked-questions


I'd grab them with a big ol' pliers and bend them back and forth a few times, breaking them off. A recipro-saw with a fine-toothed blade would also work. 

Otherwise, if they're not too long, just press your foam onto them. A little persuasion and you can forget they were ever there. The lost insulation value is trivial.