Finder "Show Icon Preview" for certain filetypes?
It's really one way or the other as you have speculated.  Don't worry, it really shouldn't slow things down having these otherwise convenient little thumbnails created for everything you have.