title: iOS CVImageBuffer distorted from AVCaptureSessionDataOutput with AVCaptureSessionPresetPhoto
body: <p>At a high level, I created an app that lets a user point his or her iPhone camera around and see video frames that have been processed with visual effects. Additionally, the user can tap a button to take a freeze-frame of the current preview as a high-resolution photo that is saved in their iPhone library.</p>
  
  <p>To do this, the app follows this procedure:</p>
  
  <p>1) Create an AVCaptureSession</p>
  
  <pre><code>captureSession = [[AVCaptureSession alloc] init];
  [captureSession setSessionPreset:AVCaptureSessionPreset640x480];
  </code></pre>
  
  <p>2) Hook up an AVCaptureDeviceInput using the back-facing camera.</p>
  
  <pre><code>videoInput = [[[AVCaptureDeviceInput alloc] initWithDevice:backFacingCamera error:&amp;error] autorelease];
  [captureSession addInput:videoInput];
  </code></pre>
  
  <p>3) Hook up an AVCaptureStillImageOutput to the session to be able to capture still frames at Photo resolution.</p>
  
  <pre><code>stillOutput = [[AVCaptureStillImageOutput alloc] init];
  [stillOutput setOutputSettings:[NSDictionary
      dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA]
      forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
  [captureSession addOutput:stillOutput];
  </code></pre>
  
  <p>4) Hook up an AVCaptureVideoDataOutput to the session to be able to capture individual video frames (CVImageBuffers) at a lower resolution</p>
  
  <pre><code>videoOutput = [[AVCaptureVideoDataOutput alloc] init];
  [videoOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
  [videoOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];
  [captureSession addOutput:videoOutput];
  </code></pre>
  
  <p>5) As video frames are captured, the delegate's method is called with each new frame as a CVImageBuffer:</p>
  
  <pre><code>- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
  {
      CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
      [self.delegate processNewCameraFrame:pixelBuffer];
  }
  </code></pre>
  
  <p>6) Then the delegate processes/draws them:</p>
  
  <pre><code>- (void)processNewCameraFrame:(CVImageBufferRef)cameraFrame {
      CVPixelBufferLockBaseAddress(cameraFrame, 0);
      int bufferHeight = CVPixelBufferGetHeight(cameraFrame);
      int bufferWidth = CVPixelBufferGetWidth(cameraFrame);
  
      glClear(GL_COLOR_BUFFER_BIT);
  
      glGenTextures(1, &amp;videoFrameTexture_);
      glBindTexture(GL_TEXTURE_2D, videoFrameTexture_);
      glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
      glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
      glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
      glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  
      glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(cameraFrame));
  
      glBindBuffer(GL_ARRAY_BUFFER, [self vertexBuffer]);
      glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, [self indexBuffer]);
  
      glDrawElements(GL_TRIANGLE_STRIP, 4, GL_UNSIGNED_SHORT, BUFFER_OFFSET(0));
  
      glBindBuffer(GL_ARRAY_BUFFER, 0);
      glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
      [[self context] presentRenderbuffer:GL_RENDERBUFFER];
  
      glDeleteTextures(1, &amp;videoFrameTexture_);
  
      CVPixelBufferUnlockBaseAddress(cameraFrame, 0);
  }
  </code></pre>
  
  <p>This all works and leads to the correct results. I can see a video preview of 640x480 processed through OpenGL. It looks like this:</p>
  
  <p><img src="https://i.stack.imgur.com/WPGdM.png" alt="640x480 Correct Preview"></p>
  
  <p>However, if I capture a still image from this session, its resolution will also be 640x480. I want it to be high resolution, so in step one I change the preset line to:</p>
  
  <pre><code>[captureSession setSessionPreset:AVCaptureSessionPresetPhoto];
  </code></pre>
  
  <p>This correctly captures still images at the highest resolution for the iPhone4 (2592x1936).</p>
  
  <p>However, the video preview (as received by the delegate in steps 5 and 6) now looks like this:</p>
  
  <p><img src="https://i.stack.imgur.com/Vt39N.png" alt="Photo preview incorrect"></p>
  
  <p>I've confirmed that every other preset (High, medium, low, 640x480, and 1280x720) previews as intended. However, the Photo preset seems to send buffer data in a different format.</p>
  
  <p>I've also confirmed that the data being sent to the buffer at the Photo preset is actually valid image data by taking the buffer and creating a UIImage out of it instead of sending it to openGL:</p>
  
  <pre><code>CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  CGContextRef context = CGBitmapContextCreate(CVPixelBufferGetBaseAddress(cameraFrame), bufferWidth, bufferHeight, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
  CGImageRef cgImage = CGBitmapContextCreateImage(context); 
  UIImage *anImage = [UIImage imageWithCGImage:cgImage];
  </code></pre>
  
  <p>This shows an undistorted video frame.</p>
  
  <p>I've done a bunch of searching and can't seem to fix it. My hunch is that it's a data format issue. That is, I believe that the buffer is being set correctly, but with a format that this line doesn't understand:</p>
  
  <pre><code>glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(cameraFrame));
  </code></pre>
  
  <p>My hunch was that changing the external format from GL_BGRA to something else would help, but it doesn't... and through various means it looks like the buffer is actually in GL_BGRA.</p>
  
  <p>Does anyone know what's going on here? Or do you have any tips on how I might go about debugging why this is happening? (What's super weird is that this happens on an iphone4 but not on an iPhone 3GS ... both running ios4.3)</p>