Transfer learning VGGish (AudioSet). Impact of zero padding to fit the input size
Will this have any negative consequences on the ability of the network to learn using my dataset?


That's really hard to say, but I suspect it does not make a difference for any sounds that aren't exclusively defined by higher frequencies.

As a rule of thumb: If a human can still correctly identify a sound sampled at 8kHz, so should a machine. If the machine cannot do it, because of the missing higher frequencies, it may not have learned the characteristics of the sound to identify in the first place, but something else.


  Do I also need to finetune the ConvLayers earlier on in the network?


I'd start with not re-training those. Having been trained on AudioSet, they should be fairly universal.


  Any other thoughts on how I can work around this (potential) issue?It is not feasible to increase the frequency of the recordings, as they are being streamed from low power devices.


Not really. You could upsample your audio to 22.05kHz and go from there—but you cannot create new information for those higher bands out of thin air. So, I don't recommend upsampling.


  Suggestions on other pretrained networks are also welcome!


Read Look, Listen, and Learn More: Design Choices for Deep Audio Embeddings by Cramer, Wu, Salamon, and Bello. They offer an open, pre-trained embedding model for environmental sounds at https://github.com/marl/openl3 (which, IIRC, is better than VGGish).

You might be able use its output as input for a shallow dense network to train a classifier for your audio (unless of course you way too few samples).