Perspective Raytracing
Let the horizontal FOV be given as theta (in radians), and the vertical FOV be given as phi (in radians). The we know that:

$$\tan\frac{\theta}{2}=\frac{w}{f}, \tan\frac{\phi}{2}=\frac{h}{f}$$

Let us pick $f = 1$, then $w = \tan\frac{\theta}{2}, h = \tan\frac{\phi}{2}$. Now your ray generation direction generation formula looks like:

$$\pmb{d} = x w\pmb{u} + yh\pmb{v} + \pmb{w}$$

Where $\pmb{u},\pmb{v}, \pmb{w}$ are your sx,sy, forward vectors. Essentially, I found the extent of the smallest/largest x and y respectively ($1w = w, -1w=-w$, etc.). You can premultiply $\pmb{u}, \pmb{v}$ with $w, h$ respectively, if you don't want to perform the multiplication per ray.