How to stop evaluation phase in reinforcement learning with epsilon-greedy Monte Carlo agent?
I do not understand, how to stop the evaluation here, because the policy update is linked to epsilon. So just setting epsilon equal to zero at some point does not seem to make sense to me.


If by evaluation you mean the act of exploring new paths, it does not need to stop by changing epsilon to 0 instantly. Instead, in order to facilitate the convergence of the algorithm, the epsilon can be progressively decreased until it reaches 0.

I do not think your question is meant to be related with monte carlo specifically, but if this was not what you wanted to know, please comment.


  But would not a epsilon value of 0 lead to an unintended policy update?


No, a value of epsilon equal to 0 will make you always choose your action according to the policy. I think what is confusing is that the "update" they make to the policy on the last for, is not an update. It is a statement saying the probability of taking such action. What it means is the following:

For action $a$, the probability of taking $a$ in state $S_t$, when $\epsilon$ is 0 is: 


1 if action $a$ is the action with the best value
0 if action  $a$ is not the action with the best value


What this means is that when epsilon is 0, the action taken will always be the greedy action.