What is the value of a state when there is a certain probability that agent will die after each step? We assume infinite horizon and discount factor $\gamma = 1$. At each step, after the agent takes an action and gets its reward, there is a probability $\alpha = 0.2$, that agent will die. The assumed maze looks like this



Possible actions are go left, right, up, down or stay in a square. The reward has a value 1 for any action done in the square (1,1) and zero for actions done in all the other squares.

With this in mind, what is the value of a square (1,1)?

The correct answer is supposed to be 5, and is calculated as $1/(1\cdot 0.2) = 5$. But why is that? I didn't manage to find any explanation on the net, so I am asking here.
The value of a state depends on the policy that you use, so I'll make the assumption here that you're talking about value using the optimal policy. 

According to the optimal policy, the agent would choose to stay in the square (1,1) every time, but since it has a 0.8 probability of actually staying (and 0.2 probability of dying), we can compute the value of the agent using the Bellman equation as:

$$
V(1,1) = 1 + 0.8 V(1,1) + 0.2 V(\text{death state}) \\
\implies V(1,1) = 1 + 0.8 V(1,1) \\
\implies V(1,1) = \frac{1}{1 - 0.8} \\
\implies V(1,1) = 5
$$

There are other ways of deriving the same number (value function has multiple definitions) but they are equivalent.