How to seal around ceiling duct?
I would use drywall compound and fill in between the wall and the duct. And then I would tape using a UL181 tape (I like foil mastic) over the inside edge of the duct and the wall. Apply the tape flat and rub it smooth. The tape will be covered by the register.

Using UL181 tape to seal the duct to the ceiling membrane really should be standard.