Title: Tell libavcodec/ffmpeg to drop frame
Tags: c;ffmpeg;video-encoding;libavcodec
Question: I'm building an app in which I create a video.
Problem is, sometime (well... most of the time) the frame acquisition process isn't quick enough.

What I'm currently doing is to skip the current frame acquisition if I'm late, however FFMPEG/libavcodec considers every frame I pass to it as the next frame in line, so If I drop 1 out of 2 frames, a 20seconds video will only last 10. More problems come in as soon as I add sound, since sound processing is way faster...

What I'd like would be to tell FFMPEG : "last frame should last twice longer that originally intended", or anything that could allow me to process in real time.

I tried to stack the frames at a point, but this ends up killing all my memory (I also tried to 'stack' my frames in the hard drive, which was way to slow, as I expected)

I guess I'll have to work with the pts manually, but all my attempts have failed, and reading some other apps code which use ffmpeg, such as VLC, wasn't of a great help... so any advice would be much appreciated!

Thanks a lot in advance!



Here is another answer: your output will probably be considered variable framerate (vfr), but you can simply generate a timestamp using wallclock time when a frame arrives and apply it to your AVFrame before encoding it. then the frame will be displayed at the correct time on playback.

for an example of how to do this (at least the specifying your own timestamp part), see doc/examples/muxing.c in the ffmpeg distribution (line 491 in my current git pull):

```frame-&gt;pts += av_rescale_q(1, video_st-&gt;codec-&gt;time_base, video_st-&gt;time_base);
```

here the author is incrementing the frame timestamp by 1 in the video codec's timebase rescaled to the video stream's timebase, but in your case you can simply rescale the number of seconds since you started capturing frames from an arbitrary timebase to your output video stream's timebase (as in the above example). for example, if your arbitrary timebase is 1/1000, and you receive a frame 0.25 seconds since you started capturing, then do this:

```AVRational my_timebase = {1, 1000};
frame-&gt;pts = av_rescale_q(250, my_timebase, avstream-&gt;time_base);
```

then encode the frame as usual.


Here is another answer: Many (most?) video formats don't permit leaving out frames. Instead, try reusing old video frames when you can't get a fresh one in time.

Comment for this answer: Most containers actually DO support leaving out frames.  MP4 only stores a duration for each frame, which doesn't have to be the same for all frames.  mkv stores presentation timestamps.  IDK about avi; maybe not, but AVI isn't ideal for modern codecs anyway.  (there aren't that many general-purpose video containers, not counting obscure ones like ogm, or REALLY obscure ones like nut.)

Here is another answer: There's this ffmpeg command line switch ```-threads ...``` for multicore processing, so you should be able to do something similar with the API (though I have no idea how). This might solve your problem.


Here is another answer: Just an idea.. when it's lagging with the processing have you tried to pass to it the same frame again (and drop the current one)? Maybe it can process the duplicated frame quickly.

Comment for this answer: Well it improves a bit, but the video is still way too fast...
I guess this comes from my scheduling algorithm (which I can't call an algorithm to be trully honest...)
I'll try to improve it tomorrow and post back if this improves something.
Thanks for the help anyhow (both you and duskwuff)!

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 4, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2011:00:07/05/11', 'FavoriteCount': 1.0, 'PostTypeId': 1, 'Tags': '<c><ffmpeg><video-encoding><libavcodec>', 'Title': 'Tell libavcodec/ffmpeg to drop frame', 'ViewCount': 3252, 'id': 6589088, 'user_AboutMe': '', 'user_AccountId': 168454.0, 'user_DisplayName': 'chouquette', 'user_Reputation': 951.0, 'user_Views': 33.0, 'user_id': 393922.0}}