<p dir="auto">There have been a few time that we've come across the situation where we want to replace nodes instead of updating in place. We are also considering moving to a side by side upgrade deployment system.</p>
<p dir="auto">The status quo makes it difficult to replace the entire cluster efficiently. We have to spin up nodes, wait for rebalancing, kill an old node, wait for rebalancing, repeat.</p>
<p dir="auto">It would be nice to be able to mark a node as retiring or soon to be deprecated so that its data would get replicated on the new nodes sooner. So, you could mark a node this way, and it would still be used for the segments it has, but it they would not be considered in the replica count.</p>
<p dir="auto">This ask is that historical nodes are able to be put into a maintenance mode such that their nodes do not count towards the replication threshold. This allows for scaling tiers down in a more controlled manner.</p>
<p dir="auto">We occasionally have capacity issues as we scale our cluster such that a lower tier fills up, and upper tiers need to be expanded to accommodate the handoff of realtime indexing tasks while the lower tiers expand. Once this is completed and segments are flowing as expected through the tiers, the "more recent data" tiers need to be shrunk back to normal operating size.</p>
<p dir="auto">Being able to put a historical node into a state where it doesn't count towards replication would help in this.</p>