Deep Learning with Spectrograms for sound recognition
RNNs were not producing good enough results and are also hard to train so I went with CNNs. 

Because a specific animal sound is only a few seconds long we can divide the spectrogram into chunks. I used a length of 3 seconds. We then perform classification on each chunk and average the outputs to create a single prediction per audio file. This works really well and is also simple to implement.

A more in-depth explanation can be found here:
http://ceur-ws.org/Vol-1609/16090547.pdf