Is staring into the cover sheet of the exam paper and looking at the second page considered cheating?
I don't know if I'm simply overthinking this or if it is a genuine ethical issue.


You're overthinking this. You are complying with all instructions, and I presume are not even intentionally exploiting any loopholes. To the extent that simply looking at the cover page allows you to guess the subject of the first question, that is a defect (if it can even be called that) of the exam.

A good exercise to test for ethics is to consider what the reaction be if you were publicly known to have "stared at" the cover sheet. Can you imagine anyone at a disciplinary board doing anything other than laughing at the triviality of the "offense"? 

Whether you should attempt to avoid seeing through the cover sheet or whether it is "fair game" is perhaps a more complex question -- however, I suggest you find more significant matters to spend your mental energies on.