Frequency Modulation to arbitrary Audio signals
I have, in the past, used delay modulation on arbitrary audio signals.  but the modulation waveform itself was not arbitrary, but was a sinusoid.  but we could speed this modulation oscillator up to an audio rate.

Now that I think of it, I tried hooking that modulation oscillator up to a pitch detector.  This is a quarter century ago, so I don't remember everything.

I s'pose you could derive that delay modulation signal from the input and likely LPF it before applying it to the instantaneous delay of the signal going out.