Finding specific spots in an audio signal In theory, how would one process an audio signal to find the spots marked in the linked picture below.

Mel Spectrogram

Unfortunately I am not really conform with signal processing and could not find any fitting solutions while looking for similar problems.
I have read a lot about signal processing but might have looked in the wrong places.

Currently, my idea would be: windowing over the signal, applying FFT to each window and check if the frequency/signal strength exceeds a certain value. I am not quite sure if this would work out, plus it seems rather ineficcient.

Can anyone point me in the direction of where to begin and how to identify these spots?
What you are describing is exactly how to plot a spectrogram. Finding the spots can be done with a classic peak-finding function in any software.

If you are looking for coding the algorithm from scratch, here is a method, which is the most intuitive but not the fastest in term of calculations :

Consider your spectrogram as a $n\times m$ array of numbers, with each number position referenced as $(i,j)$. You have to find which points are greater or equal to all of their neighbors.

In pseudo-code it would give something like :

for i from 1 to n
  for j from 1 to m
     if : 
       number(i,j) &gt; number (i-1,j-1)
   and number(i,j) &gt; number (i-1,j)
   and number(i,j) &gt; number (i-1,j+1)
   and number(i,j) &gt; number (i,j-1)
   and number(i,j) &gt; number (i,j+1)
   and number(i,j) &gt; number (i+1,j-1)
   and number(i,j) &gt; number (i+1,j)
   and number(i,j) &gt; number (i+1,j+1)
     then : 
       save (i,j) as a peak value
   end if
end for
end for


Of course, this is not very sharp since it does not consider whether you are finding a "big" peak or just a smaller one. In that case, you can improve it by setting a minimum offset value like :

if number(i,j) &gt; number(i-1,j-1) + offset


Note that this is just a simple example of a peak finding algorithm and I didn't took the boundary points specific case into account. More efficient methods exist but I can detail each one here.