Data format from recording using Audio Queue framework
<p>I'm writing an iPhone app which should record the users voice, and feed the audio data into a library for modifications such as changing tempo and pitch.  I started off with the SpeakHere example code from Apple:</p>

<p><a href="http://developer.apple.com/library/ios/#samplecode/SpeakHere/Introduction/Intro.html" rel="nofollow">http://developer.apple.com/library/ios/#samplecode/SpeakHere/Introduction/Intro.html</a></p>

<p>That project lays the groundwork for recording the user's voice and playing it back.  It works well.</p>

<p>Now I'm diving into the code and I need to figure out how to feed the audio data into the SoundTouch library (http://www.surina.net/soundtouch/) to change the pitch.  I became familiar with the Audio Queue framework while going through the code, and I found the place where I receive the audio data from the recording.</p>

<p>Essentially, you call <code>AudioQueueNewInput</code> to create a new input queue.  You pass a callback function which is called every time a chunk of audio data is available.  It is within this callback that I need to pass the chunks of data into SoundTouch.</p>

<p>I have it all setup, but the noise I play back from the SoundTouch library is very staticky (it barely resembles the original).  If I don't pass it through SoundTouch and play the original audio it works fine.</p>

<p>Basically, I'm missing something about what the actual data I'm getting represents.  I was assuming that I am getting a stream of <code>short</code>s which are samples, 1 sample for each channel.  That's how SoundTouch is expecting it, so it must not be right somehow.</p>

<p>Here is the code which sets up the audio queue so you can see how it is configured.  </p>

<pre><code>void AQRecorder::SetupAudioFormat(UInt32 inFormatID)
{
memset(&amp;mRecordFormat, 0, sizeof(mRecordFormat));

UInt32 size = sizeof(mRecordFormat.mSampleRate);
XThrowIfError(AudioSessionGetProperty(kAudioSessionProperty_CurrentHardwareSampleRate,
                                          &amp;size, 
                                          &amp;mRecordFormat.mSampleRate), "couldn't get hardware sample rate");

size = sizeof(mRecordFormat.mChannelsPerFrame);
XThrowIfError(AudioSessionGetProperty(kAudioSessionProperty_CurrentHardwareInputNumberChannels, 
                                          &amp;size, 
                                          &amp;mRecordFormat.mChannelsPerFrame), "couldn't get input channel count");

mRecordFormat.mFormatID = inFormatID;
if (inFormatID == kAudioFormatLinearPCM)
{
    // if we want pcm, default to signed 16-bit little-endian
    mRecordFormat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked;
    mRecordFormat.mBitsPerChannel = 16;
    mRecordFormat.mBytesPerPacket = mRecordFormat.mBytesPerFrame = (mRecordFormat.mBitsPerChannel / 8) * mRecordFormat.mChannelsPerFrame;
    mRecordFormat.mFramesPerPacket = 1;
}
}
</code></pre>

<p>And here's part of the code which actually sets it up:</p>

<pre><code>    SetupAudioFormat(kAudioFormatLinearPCM);

    // create the queue
    XThrowIfError(AudioQueueNewInput(
                                  &amp;mRecordFormat,
                                  MyInputBufferHandler,
                                  this /* userData */,
                                  NULL /* run loop */, NULL /* run loop mode */,
                                  0 /* flags */, &amp;mQueue), "AudioQueueNewInput failed");
</code></pre>

<p>And finally, here is the callback which handles new audio data:</p>

<pre><code>void AQRecorder::MyInputBufferHandler(void *inUserData,
                                  AudioQueueRef inAQ,
                                  AudioQueueBufferRef inBuffer,
                                  const AudioTimeStamp *inStartTime,
                                  UInt32 inNumPackets,
                                  const AudioStreamPacketDescription *inPacketDesc) {
AQRecorder *aqr = (AQRecorder *)inUserData;
try {
        if (inNumPackets &gt; 0) {
            CAStreamBasicDescription queueFormat = aqr-&gt;DataFormat();
            SoundTouch *soundTouch = aqr-&gt;getSoundTouch();

            soundTouch-&gt;putSamples((const SAMPLETYPE *)inBuffer-&gt;mAudioData,
                                   inBuffer-&gt;mAudioDataByteSize / 2 / queueFormat.NumberChannels());

            SAMPLETYPE *samples = (SAMPLETYPE *)malloc(sizeof(SAMPLETYPE) * 10000 * queueFormat.NumberChannels());
            UInt32 numSamples;
            while((numSamples = soundTouch-&gt;receiveSamples((SAMPLETYPE *)samples, 10000))) {
                // write packets to file
                XThrowIfError(AudioFileWritePackets(aqr-&gt;mRecordFile,
                                                    FALSE,
                                                    numSamples * 2 * queueFormat.NumberChannels(),
                                                    NULL,
                                                    aqr-&gt;mRecordPacket,
                                                    &amp;numSamples,
                                                    samples),
                              "AudioFileWritePackets failed");
                aqr-&gt;mRecordPacket += numSamples;
            }
            free(samples);
        }

        // if we're not stopping, re-enqueue the buffe so that it gets filled again
        if (aqr-&gt;IsRunning())
            XThrowIfError(AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL), "AudioQueueEnqueueBuffer failed");
} catch (CAXException e) {
    char buf[256];
    fprintf(stderr, "Error: %s (%s)\n", e.mOperation, e.FormatError(buf));
}
}
</code></pre>

<p>You can see that I'm passing the data in <code>inBuffer-&gt;mAudioData</code> to SoundTouch.  In my callback, what exactly are the bytes representing, i.e. how do I extract samples from <code>mAudioData</code>?</p>
