Do I have to remove the old thinset before installing new tile?
Like Greebo says, you want to get back down to the concrete, but I'd suggest that more to have a flat surface to build on top of. To make the job go faster, you can use a power tool. Several would get the job done:

Power Chisel (best fit for the job):



Air Hammer (you'd need a high capacity air compressor):



Demo Hammer (more power than you need, but it could be useful for other jobs in the future):



MC Hammer (don't touch this):