Going below the bottom of the stack in ATmega328P
I don't know about AVR specifically, but in the general case, the MCU just keeps going and write out of bounds. This is a stack overflow/underflow. (Stack underflow can only happen in assembler, not in pure C.) Typically only the linker knows where your stack begins and ends: neither the compiler nor the MCU itself knows or cares.
Generally low-end MCUs only offer physical memory access, so there's no MMU to slap your fingers if touching memory that you shouldn't touch. Or if there is a MMU, it is often a simplistic one. Higher end parts would generate hardware exceptions.
The hardened embedded systems programmers therefore ensure to always map their stack so that it overflows into harmless memory. Such as addresses where nothing is allocated, or addresses where there's nothing but unprogrammed flash. A stack pointer ending up there will quite soon lead to run-away code or similar, often trying to execute op code 0xFF. From there on you can typically catch illegal op code interrupts. Or in worst case hopefully the watchdog will stop the program.
If you don't place your stack with such care, or in case you use some default memory layout provided by the manufacturer and/or monkeys, you might end up stack overflowing into your hardware register map or some .data/.bss RAM segment, which is a disaster.
Another somewhat crude technique you can use, often done by high end system compilers, is to use a &quot;stack canary&quot;. Make a calculation of your expected worst case stack usage. Then set the whole stack to a known value at start-up (all 0x00 or all 0xFF). Then beyond your estimated worst case stack scenario, write a different value (0xAA or whatever) to various fixed locations. You can then repeatedly check these locations from the main loop to see that they still hold the fixed value - if not, then you have a stack overflow. This isn't 100% secure but catches most stack overflows.