Could symmetric keys be of sizes other than 2^x or 2^x + 2^y? Is this an issue for, perhaps, RSA? I wonder if a key could be any other size, say 261 bits.
Modern CPUs are typically 64-bit. This gives us a very good performance incentive to design symmetric cryptography with block size as a multiple of this.
There is a lesser (IMHO) drive for using $2^x$ bits, many parts especially in hardware will require ceil(log(length)) time/depth/gates by using $2^x$ we get more efficient implementations. 
Obviously we could build a less efficient (bytes/second or bytes/$) cipher with arbitrary block size.