Can I use a gap filler/expanding foam around HVAC vent?
It would be nice to know if it's a supply duct (bringing warmed or cooled air from the furnace/AC) or a return duct (taking air back to be processed again).  It looks like the latter.   

But in either case, even a supply duct is not so hot that you could not seal the opening with ordinary expanding foam.  I would do that.

Typically supply ducts are covered with some minimal insulation so their air makes it to the room without loosing the temp difference (possibly to the basement as you indicate).   Return ducts are typically not.

You could also pay attention to any leaks in the ducting assembly, and use metal tape or duct mastic to seal them.  Expanding foam would also work around the duct's opening in the floor.