Atmel SAME70 SDRAM Memory Mapping
It seems from their schematic and table 4-17 from your provided link that Address 0 of the MCU is used to control the bank selection from the memory. So that's why the address pins of the MCU are offset as you noticed.