How to search for a sound in a video
You'd probably want to download them, and process the audio, so that you could search faster than realtime.  Doing that without a paid, premium subscription might violate YouTube's Terms of Service, and it might even violate the TOS WITH a premium subscription, so be sure you study those carefully, first.
If you can manage to download them, you might be able to use fast Fourier analysis to fingerprint the sound, and then search the files for it.  You don't necessarily need to know all of the details about the signal processing, as long as you can find a tool that'll help you do it.  I'd recommend having a look at this article as a starting point, and searching online code repositories like GitHub for something that suits your level of engagement and code familiarity.  Even if you don't have any experience coding, you can sometimes find links to helpful google colabs or jupyter notebooks installations.  Here's one I found one earlier today for music segmentation, that will analyze music from YouTube links and spit out Midi keyboard parts, just by clicking through the colab.
If you aren't able to download the files, you could still theoretically use software to analyze the stream of audio coming through your speakers, worst case, but you'll still need to find suitable tools to fingerprint it, and I'm less confident that you'd be able to find an off-the-shelf solution which works that way.