How to prove that a language is not recursively enumerable
Here are two methods.
Consider the complement
Theorem. If a language $L$ and its complement are both RE, they are both recursive.
Proof. Decide whether $w\in L$ by enumerating $L$ and its complement in parallel and accept/reject as soon as $w$ appears in one of the enumerations. $\Box$
So, if you can prove that $L$ is not recursive but its complement is RE, then $L$ is not RE.
Halting problems
Theorem. Let $\mathcal{M}$ be the class of Turing machines equipped with an oracle for the ordinary Turing machine halting problem. The halting problem for $\mathcal{M}$ is not RE.
Proof. Essentially the same as the proof that the ordinary Turing machine halting problem is not recursive. $\Box$
So, if you can reduce the halting problem for $\mathcal{M}$ to your problem, your problem is not RE.