inverse FFT when only a few time points are needed I am developing C++ code to do some modeling that will take as input a parametric model of the complex frequency response (amplitude, phase) and produce as output an impulse response, e.g. using the iFFT. I would like to compare this to a "windowed" impulse response that I have measured for a loudspeaker in a room. The measured impulse can be characterized by an initial portion that corresponds to sound traveling directly through the air to the microphone that lasts a few millisecond, after which sounds that reflect off of the surfaces in the room (floor, walls, etc.) contaminates the signal. The uncontaminated portion is only 3% of the total measured impulse. For instance if I record 16k points at 96kHz, its only the first 470 points of the impulse. I want to compare the impulse response that the iFFT generates from the frequency response model to ONLY the uncontaminated portion of the measured impulse.

I could calculate the entire impulse response from the frequency response and then just throw away 97% of the result but this seems to be very inefficient. The iFFT will be calculated many, many times (thousands probably) while my model is being optimized so I want to make sure that I can make it as efficient as possible. At this point my only option seems to be using FFTW and then just throwing away the data that is not needed (for lack of a better idea).

Is there a fast way to calculate the inverse FFT only for those 470 time points of interest, e.g. not for the entire time span that the FFT can access? I am not intimately familiar with the computation of the FFT and iFFT, so I don't have insight on the answer to this question.
What you're looking for is called a pruned DFT. In principle, it is possible to calculate a subset of outputs from a DFT using fewer mathematical operations. In practice, however, existing highly-optimized FFT implementations like FFTW are designed for full-output transforms. You'll find in many cases, unless you're only concerned with a very small proportion of the transform outputs, it is fastest to just calculate the entire thing and throw away what you don't want.

As it turns out, FFTW has a page in their documentation that talks about this specifically.

If you want to put in a lot of work, it is possible that you could obtain some performance gain by rolling your own FFT implementation for your specific case. FFTW is a general-purpose, any-size-transform library that works well in a variety of conditions, but it's possible that you could beat it if you tune something specifically for your application. Before expending this effort, I would think long and hard about whether it is truly a limiting factor in your system's performance.