What is a good method for detection of rare occurencies of speech in noisy audio data?
Detecting speech specifically is a well known problem, generally called Voice Activity Detection (VAD).
The simplest mechanisms just calculate a time-averaged ratio of energy in the speech frequencies compared to total energy, many implementations on Github of this idea. The audio codes for speech also tend to include a VAD component. A modern example can be found in libvad, based on the VAD from WebRTC project. 

More generally you can try Anomaly Detection using algorithms like Isolation Forest or LocalOutlierFactor. What is critical is to use good features as input. A good starting point would be to calculate mel-spectrogram or MFCC. The spectrogram frames should be normalized, typically by subtracting the median/mean and dividing by RMS energy.
Run the anomaly detection on individual frames of 20-50ms and then do some filtering of classifications to reduce false triggering. A simple method is to require N consecutive frames to be classified as an anomaly to consider it a real anomaly. Or to require at least P percent of frames within a larger rolling window (say 200-1000ms).

In Python, you can use librosa for audio feature extraction and scikit-learn for anomaly detection 

Note that for evaluating your method you will need to collect and label a test-set.