Should I continue to remove the cement board, or just remove the tiles?
I would probably attempt to just patch the one area and srape the rest off. Touch up with a grinder.

However, that depends on how intact the existing CBU remains after scraping off the tiles and thinset.

I would take the path of least resistance. If you try one way and it is taking way longer than the other way, switch.