Comfort Noise Insertion in VAD algorithms
You may want to try a two-pronged approach.  First, have a standard pre-packaged CN to inject (like jan and Jason R suggest) at the start of the conversation/exchange.  

As the conversation progresses, gather low-volume data, perhaps as part of your noise reduction algorithm I assume you already have, by sampling the lowest volume signal coming in between VAD end and start points. Compare your low-volume samples to your standard CN, and replace (or convolve to simply dilute your synthetic CN) if it matches some threshold or characteristic you established. All you need is some rand and normalization functions, and perhaps FFT, all of which are readily available in C (and assembly).