How do I mathematically express a set generated using two loop variables within a single for loop?
You attempt is basically correct, assuming that the for-loop is short for nested loops. You'd usually write it like 

$\qquad\displaystyle\{ (n-j)(n-k) \mid j \in \{n, \dots, 2n\}, k \in \{n, \dots, 0\}\}$.

Now we note that we can write


$k$ instead of $n-k$ (in the code, reverse the direction of the for-loop) and
$-j$ with $j \in \{0, \dots, n\}$ instead of $n-j$ with $j \in \{n, \dots, 2n\}$ (do you see why?).


So we get

$\qquad \displaystyle \{ -jk \mid j, k \in \{0, \dots, n\}\}$.



If, on the other hand, the for-loop is supposed to step j and k simultaneously, building all combinations is not what happens. Then, you'd have

$\qquad\displaystyle \{ (n-j)(n-k) \mid (j,k) \in \{ (n,n), \dots, (2n,0) \} \} $

which we can rewrite by noting that


$j \in \{n, \dots, 2n\} \iff (n-j) \in \{0, -n\}$ and
$k \in \{n, \dots, 0\} \iff (n-k) \in \{0, n\}$ and


to

$\qquad\displaystyle \{ -jk \mid (j,k) \in \{(0,0), \dots, (n,-n)\} \}$.

But now $j$ and $k$ only differ by the sign, so we can rewrite using only one variable:

$\qquad\displaystyle \{ -x^2 \mid x \in \{0, \dots, n\}\}$.