Occasionally the first byte of data from an SD card SPI read is bad
Analysis with a cheap USB logic analyzer sampling at 16MHz showed that occasionally there appeared to be only 7 clock cycles grouped together.
From the hardware VHDL, I know this to be impossible - once a byte transfer is started, it will exchange bits on the following 8 clocks.

I theorized that there was noise on the SCLK signal, causing a clock that was too fast to see by the analyzer, and too fast for the computer to pick up on, but that the SD Card did actually exchange a bit on.

So I added deglitching logic to the source of the SCLK signal, and the problem has gone away.

Side note: The host computer is a retro computer running at 4MHz. a certain amount of noise on the clock probably isn't visible to most of the components in that computer, but to my modern FPGA based add-on (which can easily run designs with fMax > 50MHz) such things suddenly become an issue.

Thanks to those who have responded.