Does CPU without register exist?
Every CPU has, at the very least, some internal registers: a program counter, a register storing flags, and in some architectures, an accumulator (though the latter isn't strictly required).

As you mention, registers are important for speed reasons. CPUs are designed with efficiency in mind. There are no drawbacks to having (external) registers, so I conjecture that all CPUs have them.