How is equation 8 derived in the paper "Self-critical sequence training for image captioning"?
First of all you made a mistake, equation 8 in the paper is defined with $\frac{\partial L(\theta)}{\partial s_t}$ not $\frac{\partial L(\theta)}{\partial\theta}$.
Loss is defined as:  

$L(\theta) = - \mathbb{E}_{w^s \sim p_{\theta}}[r(w^s)]$  

If we use definition of expectation (for discrete case): 

$\mathbb{E}[X] = \sum\limits_{i} p_i(x_i)x_i$
we get  

$L(\theta) = -\sum\limits_{w^s} p_\theta(w^s)r(w^s)$  

probability $p_\theta(w_t)$ is defined as $softmax(s_t)$ = $S(s_t)$. If we calculate derivative of $L(\theta)$ with respect to $s_t$, we are basically calculating derivative of softmax $S(s_t)$ with respect to $s_t$.  

It can be shown and i won't derive it here that derivative of softmax function is  

$\frac{\partial S(s_i)}{\partial s_j} = S(s_i)(1_{ij} - S(s_j))$  

where $1_{ij}$ here is defined as yours. If we plug this derivative into derivative of  $L(\theta)$ and replace $S(s_t)$ with $p_\theta(w_t)$ (remember that probability distribution is defined as softmax here) we get:  

$\frac{\partial L(\theta)}{\partial s_t} = -\sum\limits_{w^s} p_\theta(w^s)(1_{w^s_t} - p_\theta(w_t))r(w^s)$  

we extract the - from the equation above, use definition of expectation again and subtract baseline from reward and we have:  

$\frac{\partial L(\theta)}{\partial s_t} = \mathbb{E}_{w^s \sim p_{\theta}}[(p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)]$  

now, if we ignore expectation we get:  

$\frac{\partial L(\theta)}{\partial s_t} \approx (p_\theta(w_t) - 1_{w^s_t})(r(w^s) - b)$  

I probably butchered notation is some places, but base idea should be clear