How to remove plastic junction box
It probably has wings with framing nails in them, like this:



You can either try sliding a saw through the gap outside the box and cutting the wings off, or you can cut up and remove the box a piece at a time. That's more difficult with plastic than with fiber, so be prepared for a battle. A rotary tool with a spiral cutter would work well.

If you're patching the drywall anyway, just put a block of wood in the box and drive it upward with a hammer.