Rebuild indexes on replication server
According to official advice from Microsoft there is no need to remove the subscriber.

How do I rebuild indexes or tables in replicated databases?
There are a variety of mechanisms for rebuilding indexes. They can all be used with no special considerations for replication, with the following exception: primary keys are required on tables in transactional publications, so you cannot drop and recreate primary keys on these tables.
How do I add or change indexes on publication and subscription databases?
Indexes can be added at the Publisher or Subscribers with no special considerations for replication (be aware that indexes can affect performance). CREATE INDEX and ALTER INDEX are not replicated, so if you add or change an index at, for example, the Publisher, you must make the same addition or change at the Subscriber if you want it reflected there.

There is a potential, AFAIK, if you're using transactional replication that you will get a big queue of transactions that can't be applied to your subscriber (assuming that your publisher is quite busy otherwise you wouldn't be using replication). This may need to be managed, possibly by doing a REORGANIZE instead of a REBUILD or some such, but you can gauge that best for your situation.