Optimizing the performance of a nested for loop over millions of messages from SQLite Query - v2
You could save a lot of comparisons if you wouldn't throw away the information where the last comparison ended (at 'break'). What you do is a sort of sliding window: you start at msg[k] and compare at index k+1, k+2,...k+f until the timestamp leaves the 24h range. You do this sequentially, or you could do that with a binary search - the time reduction depends on how big the 'frequency' is on average.

My suggestion to improve deals with the next iteration: instead of comparing at least f-1 values again, do this:


drop the last starting point, i.e. frequency -= 1
skip all the values k+1...k+f as you have counted them already in the last iteration
start comparing at index k+f+1, either sequentially or with binary search. Increment the last value of 'frequency' for each valid comparison.


f in this case is available in 'frequency'.
Think of a sliding window where you advance the window by 1 to the right: drop the leftmost and start comparing one after the rightmost element. This should save you a lot of time (the more, the higher the frequency is on average).