Must a Turing machine tape be binary?
If there is just one symbol which can appear on the tape, then nothing about the tape can ever change. Thus, the tape is absolutely useless, and you are left with a finite bidirectional automaton.
However, the blank symbol $\bot$ is often treated as a special case. We could thus consider the situation where each cell on the tape can carry either $0$ or $\bot$, and where we can write a $0$ on a place previously containing a $\bot$, but we can never erase a $0$ and return that cell content to $\bot$. This is as close to unary as we can really get for a TM.
As long as we only consider decision problems, these machines still have the full power. Essentially, the idea is to insert a bunch of auxiliary bits between the bits containing the actual content, and then so simply copy everything to a fresh chunk of tape whenever we need to erase a bit, except for the bit to be erased.
Having a finite number of tape symbols greater than 1 doesn't make much a difference. So binary is just special here because its the smallest alphabet that easily works.