title: How to pipe live video frames from ffmpeg to PIL?
body: <p>I need to use ffmpeg/avconv to pipe jpg frames to a python PIL (<em>Pillow</em>) Image object, using gst as an intermediary*. I've been searching everywhere for this answer without much luck. I think I'm close - but I'm stuck. <em>Using Python 2.7</em></p>
  
  <p>My ideal pipeline, launched from python, looks like this:</p>
  
  <ol>
  <li><strong>ffmpeg/avconv</strong> (as h264 video)</li>
  <li><em>Piped -></em></li>
  <li><strong>gst-streamer</strong> (frames split into jpg)</li>
  <li><em>Piped -></em></li>
  <li><strong>Pil Image Object</strong></li>
  </ol>
  
  <p>I have the first few steps under control as a single command that writes .jpgs to disk as furiously fast as the hardware will allow. </p>
  
  <p>That command looks something like this: </p>
  
  <pre><code>command = [
          "ffmpeg",
          "-f video4linux2",
          "-r 30",
          "-video_size 1280x720",
          "-pixel_format 'uyvy422'",
          "-i /dev/video0",
          "-vf fps=30",
          "-f H264",
          "-vcodec libx264",
          "-preset ultrafast",
          "pipe:1 -",
          "|", # Pipe to GST
          "gst-launch-1.0 fdsrc !",
          "video/x-h264,framerate=30/1,stream-format=byte-stream !",
          "decodebin ! videorate ! video/x-raw,framerate=30/1 !",
          "videoconvert !",
          "jpegenc quality=55 !",
          "multifilesink location=" + Utils.live_sync_path + "live_%04d.jpg"
        ]
  </code></pre>
  
  <p>This will successfully write frames to disk if ran with popen or os.system.</p>
  
  <p>But instead of writing frames to disk, I want to capture the output in my subprocess pipe and read the frames, as they are written, in a file-like buffer that can then be read by PIL. </p>
  
  <p>Something like this:</p>
  
  <pre><code>    import subprocess as sp
      import shlex
      import StringIO
  
      clean_cmd = shlex.split(" ".join(command))
      pipe = sp.Popen(clean_cmd, stdout = sp.PIPE, bufsize=10**8)
  
      while pipe:
  
          raw = pipe.stdout.read()
          buff = StringIO.StringIO()
          buff.write(raw)
          buff.seek(0)
  
          # Open or do something clever...
          im = Image.open(buff)
          im.show()
  
          pipe.flush()
  </code></pre>
  
  <p>This code doesn't work - I'm not even sure I can use "while pipe" in this way. I'm fairly new to using buffers and piping in this way. </p>
  
  <p>I'm not sure how I would know that an image has been written to the pipe or when to read the 'next' image. </p>
  
  <p>Any help would be greatly appreciated in understanding how to read the images from a pipe rather than to disk. </p>
  
  <ul>
  <li><em>This is ultimately a Raspberry Pi 3 pipeline and in order to increase my frame rates I can't (A) read/write to/from disk or (B) use a frame by frame capture method - as opposed to running H246 video directly from the camera chip.</em>  </li>
  </ul>