How to remove old-work ceiling light fixture box attached to conduit pipes
Removing this box is not a good solution when there are rigid conduits bringing wires into this existing box. The main reason for this is that those wires would still need to terminate into an accessible junction box. On top of that there is just no simple way to adapt the existing rigid conduits for connection to the floating connection box on a recessed fixture.
This may be a doable project IF:

You had access to an attic space above this ceiling
You have the experience and perseverance to work through the serious rework of the existing conduits to some other accessible junction box in the attic.
Keep in mind that the wires in those conduits may very well not be long enough when you go to try create a new junction box - which then entails pulling new wires through those conduits back to their other connection points.

The practical way to deal with this is to keep the existing ceiling boxes and install new surface mount LED fixtures. If you insist on the recessed fixtures then you will need to cut new holes in the ceiling for those fixtures near the existing electrical box and then run new flexible conduit from that existing box over to the recessed fixture floating connection box. If you choose this latter route then just purchase suitable flat white cover plates for the existing electrical box. If the ceiling is a different color the flat cover plate can be painted once installed to coordinate with the ceiling.