Interpolation of missing audio signal in a video sequence Suppose there is a video sequence and there are some frames for which the audio data is missing. I want to interpolate the missing audio data on the basis of the correlation of the audio signal with the pixels of the frames. Is there any existing research on how to do this effectively?

I was thinking of using the Wiener-Hopf equations, but I can't determine if there is any scalar parameter of the video which is correlated with the audio.

Some insight on this would help.
In general, no.  It's possible to create an MP4 file where the video frames and the audio are completely uncorrelated (edited, recorded from different source events, synthesized using different random seeds, etc.)

However, in some cases, it might be possible to infer a correlation, such as video of a human speaker whose face is clearly visible enough to allow lip reading algorithms estimate some percentage of the phonemes, with a statistically significant match against a portion of the nun-corrupted audio, etc.