Running code from different storage
when I plug a linux stick into my computer my computer is unable to execute from that stick in the same way that an avr is unable to execute from external storage.  (there are exceptions but not in the arduio case)

How it works in your computer is code that is on rom (can execute from there) runs and loads code into ram which knows how to load code from media into ram and then run it in ram.  And that code in ram can load more code into ram and run that code in ram and so on.    You would need to do a similar thing with the arduino.

Another popular approach is to use a scripty (interpreted) language, basic, python, etc.   The firmware on the avr could be the virtual machine that interprets a compiled version of the language and that firmware could read that bytecode from any media you choose.  Slow, but functions.  This is how things like the basic stamp work(ed).

It is rare but there are some mcus (dont think anything avr, but could be proven wrong) that have an sram or other interface that can be mapped into an executable memory space.  Kinda stops becoming an mcu at that point, and is usually for cases where you want to run a beefy operating system that wont fit on chip, so a processor/system that is more suited to an operating system being run on it.