How do I compute the variance of the return of an evaluation policy using two behaviour policies?
If you simulate many trajectories and receive many estimates of the two returns you're interested in, you could empirically compare their sample variances.

However, the variance of ordinary importance sampling is in general unbounded.
If you're wanting some theoretical bounds on the variance of importance sampling estimates, I'd start with weighted importance sampling, whose variance converges to zero (Sutton and Barto, section 5.5).