How to do a spectral fade out? I'd like to do a fade out of a sound in a few seconds, but in a "spectral way", with a special curve : 


(here x-axis is time, y-axis is frequency from a few hz to 20khz).

All the part "to be removed" has to be removed !

Do you know how to do this ? (an EQ / simple IIR/FIR filter  won't work for this, etc.)
You want a time-varying low pass filter, automated so that the cutoff frequency follows your curve. For such slow modulations most IIR low pass filters are stable, so you can just find a steep enough low pass filter and update its coefficients while filtering while keeping the filter state.