AVR [ Why IO Registers have IO addresses and SRAM Adresses ]
This is just a processor design choice.

You are probably surprised, because, say, on an Intel x86 processor, a register like EAX doesn't have a memory address; it lives only inside the processor.

It has to be that way on a high performance computer, because accessing memory is costly. Registers are used precisely because they are not memory: they are very high speed storage locations that are in the processor itself.

An advantage of having registers mapped to memory is that the machine state is an object in memory. If you have a snapshot of some memory range, then you have a snapshot of the machine state.  And if you have a debugging tool which can access memory, you know the machine state.

So then the second question might be: if a processor has registers which are actually just memory locations, why bother with registers at all if they are just "fake", and denote memory locations? The instruction set can just deal strictly with memory operands and no registers.

Part of the answer to that is that registers can make for a smaller instruction encoding.  If you have, say, eight registers, then only three bits are needed in an instruction to designate that register. You then have more bits available for more instructions, or more addressing modes. 

Possibly, the "machine with fake registers" can actually have registers on the processor die, which are only synchronized with the ones in memory. That is to say, it is not necessary for a read access to register R0 to actually access the memory $0000; there can still be an internal R0. To keep the internal R0 synchronized, the processor can snoop the bus and when it detects that something (possibly itself) writes to the word at location $0000, it updates the internal R0 with the same data. (Note: not saying that your AVR does this!) Similarly, if a value is loaded into R0, it can immediately go to the internal R0, but the external update of $0000 can perhaps lag behind a little bit; the machine's execution does not have to stall until that update completes. If the next instruction requires the value of R0, it does not necessarily have to wait until a memory cycle finishes updating $0000.

There are instruction sets without registers. Stack machines are an example of instruction sets which achieve good instruction size, without registers. Instead of registers, there is an stack, and most operations work with one or two operands at the top of the stack. The operations do not have to indicate the source or destination at all; the stack is implicitly used.