Will removing the X-Drupal-Cache header break anything? I have a client that wants to get rid of as much server information as possible from their headers, including "X-Drupal-Cache".

I know that anyone technical enough to be looking at headers can very likely tell that Drupal in plenty of other ways  but that's beside the point.

It is easy to remove the header but if I do will it potentially break anything?

Note: I'm aware that there is a similar question at X-Drupal-Cache http header but that issue has long ago had an accepted answer that does not cover my specific question.
X- headers are optional, and are not allowed to be officially required, except in the same family of products. You can search Internet for some bizarre examples. My favourite is X-PoweredBy: Unicorns and X-hacker: If you’re reading this, you should visit ******.com/jobs and apply to join the fun, mention this header.

That said, it may not be required, but it may still be used. If your hosting company provides caching and load balancing, it may use this header to know that page came from Drupal's cache and thus there is no need to use that data to repopulate cache server's storage. It can be also used to track statistics. and of course you are using it to test if cache works OK on production, aren't you? All these uses will get impossible if you will remove this header, or rename it without coordination with other users. Site should be served OK, but in specific circumstances there is a risk of worse performance and crippled debug/analytic options. Maybe you can afford that risk. Maybe not.

TL;DR You need to ask anyone who might be using it, not us.