I need to compute only a small number of low frequency Fourier components of a complex 2-dimensional array. I'll be computing the same Fourier components over and over again as the input array changes. Clearly, in the limit where I only want one Fourier component, it would be fastest to build a DFT matrix that gives the component I'm after, and multiply by that matrix repeatedly.

In the other limit, if I wanted all Fourier components, it would be faster to use an FFT.

At what point does it become faster to compute the FFT of the array and simply pull out the components that I'm after?

If it makes a difference, in my particular situation the input array will be something like $256\times256$. I'm using MATLAB, so that means my FFT is done using FFTW, and a matrix multiplication for a matrix DFT is done via whatever matrix multiply algorithm MATLAB uses under the hood.