Converting mel spectrogram to spectrogram
Both taking a magnitude spectrogram and a Mel filter bank are lossy processes.  Important information needed to reconstruct the original will have been lost.  Thus you need to go back and use the original audio samples to do the reconstruction by determining a time or frequency domain filter equivalent to your dimensionality reduction.

You can make assumptions about the lost information, but those assumptions themselves usually sound inaccurate, artificial and/or robotic.  Or you can use only specially synthesized input, where the assumptions will be correct by design of that input.