Does adding a constant to all rewards change the set of optimal policies in episodic tasks?
Generally we can write for $R_c$ the total reward with added constant $c$ of a policy as
$$
R_c = \sum_{i=0}^K (r_i + c) \gamma^i = \sum_{i=0}^K r_i \gamma^i + \sum_{i=0}^K c \gamma^i
$$
So if we have two policies with the same total reward (without added constant)
$$
\sum_{i=0}^{K_1} r_i^1 \gamma^i = \sum_{i=0}^{K_2} r_i^2 \gamma^i
$$
but with different lengths $K_1 \neq K_2$ the total reward with added constant will be different, because the second term in $R_c$ ( $\sum_{i=0}^K c \gamma^i$ ) will be different.

As an example: Consider two optimal policies, both generating the same cumulative reward of 10, but the first policy visits 4 states, before it reaches a terminal state, while the second visits only two states. The rewards can be written as:
$$
10 + 0 + 0 + 0 = 10
$$
and 
$$
0 + 10 = 10
$$
But when we add 100 to every reward:
$$
110 + 100 + 100 + 100 = 410
$$
and 
$$
100 + 110 = 210
$$
Thus, now the first one is better.

In the continious case, the episodes always have length $K = \infty$. Therefore, they always have the same length, and adding a constant doesnt change anything, because the second term in $R_c$ stays the same.