What does 'acting greedily' mean?
In RL, the phrase &quot;acting greedily&quot; is usually short for &quot;acting greedily with respect to the value function&quot;. Greedy local optimisation turns up in other contexts, and it is common to specify what metric is being maximised or minimised. The value function is most often the discounted sum of expected future reward, and also the metric used when defining a policy as &quot;acting greedily&quot;.
It is possible to define a policy as acting greedily with respect to immediate expected reward, but not the norm. As a special case, when the discount factor $\gamma = 0$ then it is the same as acting greedily with respect to the value function.
When rewards are sparse (a common situation in RL), acting greedily with respect to expected immediate reward is not very useful. There is not enough information to make a decision.

I want to know the actions that will be taken in 2 cases:

$v_\pi(s)$ is known and $R_s$ is also known (only).


To act greedily in RL, you would use the value function $v_\pi(s')$ - the value function of the next states. To do so, you need to know the environment dynamics - to go with the notation $R_s$ you should also know the transition matrix $P_{ss'}^a$ - the probability of transitioning from $s$ to $s'$ whilst taking action $a$:
$$\pi'(s) = \text{argmax}_a \sum_{s'} P_{ss'}^a(R_{s} + \gamma v_{\pi}(s'))$$
Notes:

This assumes $R_{s}$ is your immediate reward for leaving state $s$. Substitute $R_{s'}$ if the reward matrix is for entering state $s'$ instead.

The policy gained from acting greedily with respect to $v_\pi(s)$ is not $\pi$, it is (a usually improved) policy $\pi'$




$q_{\pi}(s, a)$ is known and $R_s^a$ is also known (only).


To act greedily in RL, you would use the value function $q_{\pi}(s, a)$, and this is much simpler:
$$\pi'(s) = \text{argmax}_a q_{\pi}(s, a)$$
Again, the policy gained from acting greedily with respect to $q_\pi(s,a)$ is not $\pi$, it is (a usually improved) policy $\pi'$