How do I fill a hole surrounding a boiler flue?
There are several spray foams in the marketplace designed specifically for this situation. I have used Great Stuff Fireblock and Pur Fill Fireblock in many commercial applications and have passed all Fire Marshal inspections. They should work well in a residential application as well. Both of these products are rated at a 240 degree flashpoint and can also be trimmed flush after drying.

Here's a neat video that shows how well these products work.