Patches of drywall mud on subfloor. Do I need to remove before installing flooring?
Just scraping it flush with the surrounding area of the subfloor will work fine. At most, I would vacuum areas where thinset is to be used under the cement board.This will pick up the dust in  the grain of the plywood subfloor, giving room for the thinset to grab better. Make sure the plywood is well fastened, 6" on edges, 8" both ways in the center of the panels. Screws are best. The diagonal subfloor hopefully was secured better first... nails reset, screws added...

In general I would vacuum everything that gets a finish over it to control future dust.