Is there a mistake in Lecture 5 of Stanford CS234 available on youtube?
It's the notation that might be a bit confusing. Take a look at David Silver slides: pages 10-15. He has a complete derivation. Do not forget that the term $r + \gamma V(s';w)$ is the target. She mentions that in the video and the fact that you are actually doing supervised learning with target provided by a bootstrapped value (you try to minimize the Bellman error).
In other words: you are trying to minimize the error between a target value and the estimation from your model. You do not know the target value so you estimate it with bootstrapping. Then you have $SE=(y - \hat{y}(w))^2$. The target y is considered known (as in supervised learning) so eventually you are searching for the weights $w$ that will make the output $\hat{y}$ of your model close to y. What I wrote here applies for batch training (minimizing the mean square error MSE).
So eventually yes the quantity $V(s';w)$ is considered constant and will have derivative 0.