Trying to identify gray crumbly filler material
Looks like concrete fill (a gypsum fiberglass product).  As Ecnerwal stated you can't really be sure without sending it to the lab.  Usually affordable ~$50 to test a sample.  Bag it, drop it off and wait for the results for piece of mind.