How can I speed up reading of an SD card?
Well I have two ideas on the SD card side, though neither really seem great to me they could still help you. I read on the Arduino forum that the SD card R/W speeds generally come out to 600kB/s, I'm not really sure why, the Arduino SPI can go up to 8 MB/s and I didn't think there would be that much protocol/data-handling in SD. 


Sector R/W - As Chris Stratton said, this is an obvious method, it's probably much faster to do sector reads. You may have to greatly change your code to implement it, but I imagine you'd see performance increase in the end. Apparently there's an unofficial library for it, though it seems to impart some potentially serious limitations.
RAM - This is the second thing that came to mind when thinking about the problem is that you could use external RAM, when not handling network transfers (Idling, should be most of the time) the Arduino transfers data to external SRAM which you add. SPI RAM is most likely the easiest to implement, but not likely to be any faster than an SD card. Parallel RAM is going to be difficult to implement, you'll likely need to write your own class/library, but it could be much faster than SPI RAM or an SD card.


I'm not actually sure if either of these would be any faster, it's all just theory and I hope others will share their thoughts. If you have trouble finding the download link for the SD Card library, it's right here. In that ZIP folder there's a file called SDCARD, just move that folder into your Arduino libraries folder on your computer.