A question on physical locations of the registers in ATmega328P When I read about registers I always encounter the following illustration:



So there are 32 registers in the above diagram. They are processor registers.

But in the data-sheet of ATmega328P at page 428 there is a section called "Register Summary".

Here in that list there are much more than 32 registers. Do the rest of the registers physically stay somewhere else? 

Lest's take an example of ADMUX register. Where is that register physically inside the DIP? It is not one of the 32 registers? And if stays somewhere else how does it connected to the ALU?
You are confusing CPU registers and SFRs, Special Function Registers.

CPU registers only exist within the CPU core. SFRs exist within the RAM bus's memory map. 

CPU registers are used by the CPU to execute instructions and do calculations, etc. SFRs are a part of the logic of the rest of the chip and directly affect the working of the various peripherals.

As far as the CPU is concerned an SFR is just a bit of SRAM, but it affects physical workings when written to, and reflects physical states when read from.