Can you decimate / downsample a signal in frequency domain just like you can interpolate / upsample it?
Your interpolation operation is an upsampling operation with subsequent low-pass filtering. This translates to zero-padding in the frequency domain. Given your original signal $x[n]$, you upsample it to become

$$y[n] = \begin{cases}x[n/M]&amp; n/M \in \mathcal{Z} \\ 0 &amp; \text{else}\end{cases}$$

to become an M-times upsampled signal. Then, your interpolated signal is 

$i[n] = h[n] * y[n]$ where $h[n]$ is a Dirichlet kernel/periodic sinc function with zeros of distance $M$. In the spectral domain, this corresponds to 

$$Y(f) = \sum_{m=0}^{M-1}X(M(f-m))=X(Mf)$$
I.e. a M-times periodic repetition of its spectrum (Note that a periodic spectrum creates the discrte time-domain signal).  Here, I assume $f=0...1$ for the discrete case and the last equality follows from periodicity of $X(f)$ here. Also note that your original spectrum is narrowed, since your sampling frequency increased.

Now, you filter out the spectrum repetitions by multiplying with a rect-function in the frequency domain. This corresponds to convolution in time with the sinc function.

Now, to do the reverse (i.e. decimation), this is what happens: Let $x[n]$ be your original signal you want to decimate. You calculate 

$y[n] = x[Mn]$ and this corresponds to the following in the spectrum:

$$Y(f) = \sum_{m=0}^{M-1} X((f-m)/M)$$

I.e. your original spectrum is repeated M times overlapping each other and summed together. This in general creates aliasing. Hence, a simple removal of higher frequencies in the DFT domain is not the same as taking every Mth sample from the time domain signal. Instead, you need to take the spectrum, sum it up repeatedly (such that it becomes M periods within your full frequency band) and finally perform the ifft of one period of the spectrum.

Your approach works, if your signal is bandlimited enough, if it does not create aliasing, i.e. the bandwidth of the signal is smaller than 1/M. Then, the shifted spectra dont overlap and you can just remove do ifft of the smaller portion. Alternatively, you can see your approach as filtering with an aliasing filter (i.e. lowpass with cutoff 1/M) before downsampling.