Architectures that take inputs of mixed sampling rates
With a search online, I couldn't find a satisfying answer, so I came up with a way. I tested and it gave pretty nice accuracy. This method is inspired by this paper.
Similar to the approach in the paper, I first select the number of convolution kernels I want to use, and depending on the sampling rate of a data point, I apply convolution to it using the convex hull of the set of convolution kernels.
So how exactly do I come up with the weight? There are a lot of hyperparameters, but what I did is assign a sampling rate to each kernel by dividing up the expected range of sampling frequency. I then subtract the sampling rate of the input datapoint to see which kernel is the closest to it. Then I take a softmin of the absolute difference, which makes sure the resulting weights add up to 1.
There are many things to play around here: Do we uniformly assign the rates? How many kernels to use in a set? How do you calculate the weight?...