How to prevent distortion after down-sampling? I am new to audio signal processing, but in one of my machine learning projects that I am working on, I need to down-sample my audio somewhere in the middle of my neural network. Therefore, I need a down-sampling method that can apply to a tensor with the shape (batch, len). (I am using PyTorch).

Since Max pooling and Average pooling are usually used to down-sample 2D images, I am thinking about if I can try my luck with them.

import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import resample
import torch
from torch.nn import MaxPool1d, AvgPool1d, AdaptiveAvgPool1d

#-------------Generating Artifical Signal -----------------
fs = 44100
s = np.linspace(0,1,fs)
x = np.sin(2*np.pi*250*s, dtype=np.float32) + np.sin(2*np.pi*280*s, dtype=np.float32) \
    + np.sin(2*np.pi*480*s, dtype=np.float32) + np.sin(2*np.pi*1600*s, dtype=np.float32)
uprange = 800
plt.figure(figsize=(8,3))
plt.plot(x[:uprange])

#------------Comparing two resampling methods ------------
torch_downsample = AvgPool1d(2,2) # Will be used for downsampling
x_torch = torch.tensor(x)[None, None, :]
for i in range(6):
    uprange = uprange//2 # the number of samples being plotted out
    x = resample(x,len(x)//2) # downsample with fft resampling algorithm
    x_torch = torch_downsample(x_torch) #downsample with Pooling

    fig, ax = plt.subplots(1,2,figsize=(16,3)) # plot out the resampled waveforms
    fig.suptitle('Resampled {} time(s)'.format(i+1), fontsize=16)
    ax[0].plot(x[:uprange], '-b')
    ax[0].plot(x_torch.view(-1).numpy()[:uprange], '--r')

    z = np.fft.fft(x) # Analyzing the spectrum
    ax[1].plot(abs(z)[:len(x)//2])

    z = np.fft.fft(x_torch[0,0,:].numpy())
    ax[1].plot(abs(z)[:len(x)//2], '--r')
    print("len1 = {}, len2 = {}".format(x.shape, x_torch.shape))
    ax[0].legend(['fft_res', 'AvgPool'])
    ax[1].legend(['fft_res', 'AvgPool'])
    plt.show()


My results



As you can see for the first 4 resamplings, the frequency components are still correct. But starting from the 5th resampling, there's a small defect at around 200Hz. The 6th resampling results in more defects.

My Questions


Is there any method to prevent these defects?
Also, am I doing downsampling correctly with Average pooling? I have
no knowledge in audio signal processing, but I am trying hard to use
my machine learning knowledge here.


Preferably, the solution can be implemented in PyTorch. (Don't tell me to use torchaudio, it messed up my whole PyTorch installation in an attempt to install it. I don't want to go through the trouble to trouble-shoot and reinstall everything)
Read up on the sampling theorem, e.g. https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem

If your feature vectors are still in the time domain and linearly derived from the original audio wave forms, you just need to low pass filter before down sampling. 

You need to low-pass filter to avoid aliasing. There many different types of lowpass filters and the best choice depends on the specific requirements of your application (causality, latency, phase distortion, transients, size of transition band, MIPS &amp; Memory, required aliasing rejection, pass band ripple tolerance, etc.) 

As you lowpass filter, you lose information. If you need to down sample to a 500 Hz sampling rate all information above 250 Hz (in practice more like 220 Hz) is lost.

Lowpass filters are can be tricky to design depending on your requirements but the code to run them is very straight forward and can easily be done from scratch. Just make sure you use cascaded second order sections and Direct From I or Transposed Form  II.