Booting from SD card in SPI mode
I'm not intimately familiar with the details of booting Linux on ARM, but I once had to boot a DSP chip from a device that the on-chip boot ROM didn't understand. I ended up writing a second-stage bootloader that the on-chip loader could read, after which my loader reconfigured the chip and booted the actual application.

You might need to do something similar, using a small SPI flash for the second-stage bootloader, and wiring your SD card to the same SPI inteface using a second chip-select line.