title: Using ffmpeg to capture frames from webcam and audio from micro and saving to file
body: <p>For the past few weeks I've been struggling with the ffmpeg API since I can not find a clear documentation and I also find it hard to search as all the solutions I find online involve not the c API but the ffmpeg.c command line program. I am creating a program which needs to capture video from a webcam and audio, show the frames on screen and record both the audio and frames to a video file. I am also using QT as a framework for this project.</p>
  
  <p>I've been able to show the frames on the screen and even record them, but my problem is the record of both the audio and video. I've decided to create a simpler program for tests, that only saves the stream to a file without showing the frames on screen, starting from the <a href="https://www.ffmpeg.org/doxygen/trunk/remuxing_8c-example.html" rel="noreferrer">remuxing.c example</a> on the ffmpeg documentation. My code is as follows:</p>
  
  <pre><code>//This is the variables on the .h
  AVOutputFormat *ofmt;
  AVFormatContext *ifmt_ctx, *ofmt_ctx;
  
  QString cDeviceName;
  QString aDeviceName;
  
  int audioStream, videoStream;
  bool done;
  
  //The .cpp
  #include "cameratest.h"
  #include &lt;QtConcurrent/QtConcurrent&gt;
  #include &lt;QDebug&gt;
  
  CameraTest::CameraTest(QString cDeviceName, QString aDeviceName, QObject *parent) :
      QObject(parent)
  {
      done = false;
      this-&gt;cDeviceName = cDeviceName;
      this-&gt;aDeviceName = aDeviceName;
      av_register_all();
      avdevice_register_all();
  }
  
  void CameraTest::toggleDone() {
      done = !done;
  }
  
  int CameraTest::init() {
      ofmt = NULL;
      ifmt_ctx = NULL;
      ofmt_ctx = NULL;
  
      QString fullDName = cDeviceName.prepend("video=") + ":" + aDeviceName.prepend("audio="); 
      qDebug() &lt;&lt; fullDName;
      AVInputFormat *fmt = av_find_input_format("dshow");
  
      int ret, i;
  
      if (avformat_open_input(&amp;ifmt_ctx, fullDName.toUtf8().data(), fmt, NULL) &lt; 0) {
         fprintf(stderr, "Could not open input file '%s'", fullDName.toUtf8().data());
         return -1;
      }
      if ((ret = avformat_find_stream_info(ifmt_ctx, 0)) &lt; 0) {
         fprintf(stderr, "Failed to retrieve input stream information");
         return -1;
      }
      av_dump_format(ifmt_ctx, 0, fullDName.toUtf8().data(), 0);
      avformat_alloc_output_context2(&amp;ofmt_ctx, NULL, NULL, "test.avi");
      if (!ofmt_ctx) {
         fprintf(stderr, "Could not create output context\n");
         ret = AVERROR_UNKNOWN;
         return -1;
      }
      ofmt = ofmt_ctx-&gt;oformat;
  
      for (i = 0; i &lt; ifmt_ctx-&gt;nb_streams; i++) {
         AVStream *in_stream = ifmt_ctx-&gt;streams[i];
         AVStream *out_stream = avformat_new_stream(ofmt_ctx, in_stream-&gt;codec-&gt;codec);
  
         if (ifmt_ctx-&gt;streams[i]-&gt;codec-&gt;codec_type == AVMEDIA_TYPE_VIDEO) {
             videoStream = i;
         }
         else if (ifmt_ctx-&gt;streams[i]-&gt;codec-&gt;codec_type == AVMEDIA_TYPE_AUDIO) {
             audioStream = i;
         }
  
         if (!out_stream) {
             fprintf(stderr, "Failed allocating output stream\n");
             ret = AVERROR_UNKNOWN;
             return -1;
         }
         ret = avcodec_copy_context(out_stream-&gt;codec, in_stream-&gt;codec);
         if (ret &lt; 0) {
             fprintf(stderr, "Failed to copy context from input to output stream codec context\n");
             return -1;
         }
         out_stream-&gt;codec-&gt;codec_tag = 0;
         if (ofmt_ctx-&gt;oformat-&gt;flags &amp; AVFMT_GLOBALHEADER)
             out_stream-&gt;codec-&gt;flags |= CODEC_FLAG_GLOBAL_HEADER;
      }
      av_dump_format(ofmt_ctx, 0, "test.avi", 1);
      if (!(ofmt-&gt;flags &amp; AVFMT_NOFILE)) {
         ret = avio_open(&amp;ofmt_ctx-&gt;pb, "test.avi", AVIO_FLAG_WRITE);
         if (ret &lt; 0) {
             fprintf(stderr, "Could not open output file '%s'", "test.avi");
             return -1;
         }
      }
      ret = avformat_write_header(ofmt_ctx, NULL);
      if (ret &lt; 0) {
         fprintf(stderr, "Error occurred when opening output file\n");
         return -1;
      }
      QtConcurrent::run(this, &amp;CameraTest::grabFrames);
      return 0;
  }
  
  void CameraTest::grabFrames() {
      AVPacket pkt;
      int ret;
      while (av_read_frame(ifmt_ctx, &amp;pkt) &gt;= 0) {
          AVStream *in_stream, *out_stream;
          in_stream  = ifmt_ctx-&gt;streams[pkt.stream_index];
          out_stream = ofmt_ctx-&gt;streams[pkt.stream_index];
          /* copy packet */
          pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream-&gt;time_base, out_stream-&gt;time_base, (AVRounding) (AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
          pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream-&gt;time_base, out_stream-&gt;time_base, (AVRounding) (AV_ROUND_NEAR_INF|AV_ROUND_PASS_MINMAX));
          pkt.duration = av_rescale_q(pkt.duration, in_stream-&gt;time_base, out_stream-&gt;time_base);
          pkt.pos = -1;
          int ret = av_interleaved_write_frame(ofmt_ctx, &amp;pkt);
          if (ret &lt; 0) {
             qDebug() &lt;&lt; "Error muxing packet";
             //break;
          }
          av_free_packet(&amp;pkt);
  
          if(done) break;
      }
      av_write_trailer(ofmt_ctx);
  
      avformat_close_input(&amp;ifmt_ctx);
      /* close output */
      if (ofmt_ctx &amp;&amp; !(ofmt-&gt;flags &amp; AVFMT_NOFILE))
         avio_close(ofmt_ctx-&gt;pb);
      avformat_free_context(ofmt_ctx);
      if (ret &lt; 0 &amp;&amp; ret != AVERROR_EOF) {
          //return -1;
         //fprintf(stderr, "Error occurred: %s\n", av_err2str(ret));
      }
  }
  </code></pre>
  
  <p>The av_interleaved_write_frame returns an error with the video packets. The end file shows only the first frame but the audio seems to be ok. </p>
  
  <p>On the console this is what is printed:</p>
  
  <pre><code>Input #0, dshow, from 'video=Integrated Camera:audio=Microfone interno (Conexant 206':
    Duration: N/A, start: 146544.738000, bitrate: 1411 kb/s
      Stream #0:0: Video: rawvideo, bgr24, 640x480, 30 tbr, 10000k tbn, 30 tbc
      Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
  Output #0, avi, to 'test.avi':
      Stream #0:0: Video: rawvideo, bgr24, 640x480, q=2-31, 30 tbc
      Stream #0:1: Audio: pcm_s16le, 44100 Hz, 2 channels, s16, 1411 kb/s
  
  [avi @ 0089f660] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
  [avi @ 0089f660] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
  [avi @ 0089f660] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 4396365 &gt;= 4396365
  [avi @ 0089f660] Too large number of skipped frames 4396359 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396360 &gt; 60000
  [avi @ 0089f660] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 4396390 &gt;= 4396390
  [avi @ 0089f660] Too large number of skipped frames 4396361 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396362 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396364 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396365 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396366 &gt; 60000
  [avi @ 0089f660] Too large number of skipped frames 4396367 &gt; 60000
  </code></pre>
  
  <p>This seems to me like a simple problem to solve but I really am mostly clueless about the ffmpeg API, if someone could lead me to the right direction that would be great!</p>
  
  <p>Thanks!</p>