Binary classification problem with imbalanced dataset, how to compare to random classifier
Since you are interested in different decision thresholds, your random model should produce scores.  In that case, a reasonable base-line model assigns a score uniformly at random in $[0,1]$.  Such a model will, at threshold $t$, have 

$$\begin{align*}
\operatorname{precision} &amp;= \frac{2\%\cdot N\cdot (1-t)}{N(1-t)} = 0.02,\\[1em]
\operatorname{recall} &amp;= \frac{2\%\cdot N\cdot (1-t)}{2\%\cdot N} = 1-t.
\end{align*}
$$ 

(Perhaps a very simple model will serve as a better baseline.)