Why is the fraction of time spent in state $s$, $\mu(s)$, not in the update rule of the parameters? I am reading &quot;Reinforcement Learning: An Introduction (2nd edition)&quot; authored by Sutton and Barto. In Section 9, On-policy prediction with approximation, it first gives the mean squared value error objective function in (9.1):
$\bar{VE}(\boldsymbol{w}) = \sum_{s \in S} \mu(s)[v_{\pi}(s) - \hat{v}(s,\boldsymbol{w})]^2$. (9.1)
$\boldsymbol{w}$ is a vector of the parameterized function $\hat{v}(s,\boldsymbol{w})$ that approximates the value function $v_{\pi}(s)$. $\mu(s)$ is the fraction of time spent in $s$, which measures the &quot;importance&quot; of state $s$ in $\bar{VE}(\boldsymbol{w})$.
In (9.4), it states an update rule of $\boldsymbol{w}$ by gradient descent:
$\boldsymbol{w}_{t+1} = \boldsymbol{w} -\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$. (9.4)
I have two questions regarding (9.4).

Why $\mu(s)$ is not in (9.4)?
Why is it the &quot;minus&quot; instead of &quot;+&quot; in (9.4)? In other words, why is it $\boldsymbol{w} -\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$ instead of $\boldsymbol{w} +\frac{1}{2}\alpha \nabla[v_{\pi}(S_t) - \hat{v}(S_t,\boldsymbol{w})]^2$?
$\mu(s)$ is not in equation (9.4) because we are assuming that the examples by which we update our parameter $w$, i.e. the frequency of which we will observe the states during online training, is the same. That is, it is a constant with respect to $w$ and since we are differentiating it can be somewhat disregarded as a constant of proportionality -- it essentially can be 'absorbed' by $\alpha$.

The minus is there because we are performing gradient descent. For more information on this, see e.g. the wikipedia page