text:
restreaming rtsp using ffmpeg delay problem
I have ip and usb cameras. I try restream rtsp from camera using ffmpeg and mediamtx (rtsp simple server). For usb camera it works fine without any delay:
mediamtx output:
anton@anton:~$ RTSP_RTSPADDRESS=:9000 ./mediamtx
INF MediaMTX v1.2.0
INF configuration loaded from /home/anton/mediamtx.yml
INF [RTSP] listener opened on :9000 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
INF [RTMP] listener opened on :1935
INF [HLS] listener opened on :8888
INF [WebRTC] listener opened on :8889 (HTTP)
INF [SRT] listener opened on :8890 (UDP)
INF [RTSP] [conn 127.0.0.1:36742] opened
INF [RTSP] [session b6d6f987] created by 127.0.0.1:36742
INF [RTSP] [session b6d6f987] is publishing to path 'camera1', 1 track (MPEG-4 Video)
INF [RTSP] [conn 192.168.3.77:50422] opened
INF [RTSP] [session cec77920] created by 192.168.3.77:50422
INF [RTSP] [session cec77920] is reading from path 'camera1', with UDP, 1 track (MPEG-4 Video)

ffmpeg pipeline output:
anton@anton:~$ ffmpeg -re -rtsp_transport tcp -i rtsp://192.168.3.100:9005/camera5 -codec copy -f rtsp rtsp://localhost:9000/camera1
ffmpeg version 3.2.18-0+deb9u1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version=0+deb9u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, rtsp, from 'rtsp://192.168.3.100:9005/camera5':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.000367, bitrate: N/A
    Stream #0:0: Video: mpeg4 (Simple Profile), yuv420p, 648x486 [SAR 1:1 DAR 4:3], 30 tbr, 90k tbn, 30 tbc
Output #0, rtsp, to 'rtsp://localhost:9000/camera1':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
    encoder         : Lavf57.56.101
    Stream #0:0: Video: mpeg4 (Simple Profile), yuv420p, 648x486 [SAR 1:1 DAR 4:3], q=2-31, 30 tbr, 90k tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
frame= 2609 fps= 30 q=-1.0 Lsize=N/A time=00:01:26.93 bitrate=N/A speed=0.999x

for ip camera the same commands work too, but restreaming comes out with 0.5 - 1 second delay:
mediamtx output:
anton@anton:~$ RTSP_RTSPADDRESS=:9000 ./mediamtx
INF MediaMTX v1.2.0
INF configuration loaded from /home/anton/mediamtx.yml
INF [RTSP] listener opened on :9000 (TCP), :8000 (UDP/RTP), :8001 (UDP/RTCP)
INF [RTMP] listener opened on :1935
INF [HLS] listener opened on :8888
INF [WebRTC] listener opened on :8889 (HTTP)
INF [SRT] listener opened on :8890 (UDP)
INF [RTSP] [conn 127.0.0.1:36736] opened
INF [RTSP] [session 967ae51e] created by 127.0.0.1:36736
INF [RTSP] [session 967ae51e] is publishing to path 'camera1', 1 track (H264)
INF [RTSP] [conn 192.168.3.77:50408] opened
INF [RTSP] [session 5b412272] created by 192.168.3.77:50408
INF [RTSP] [session 5b412272] is reading from path 'camera1', with UDP, 1 track (H264)

ffmpeg pipeline output:
anton@anton:~$ ffmpeg -re -rtsp_transport tcp -i rtsp://192.168.3.107:554/av2_0 -codec copy -f rtsp rtsp://localhost:9000/camera1
ffmpeg version 3.2.18-0+deb9u1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version=0+deb9u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, rtsp, from 'rtsp://192.168.3.107:554/av2_0':
  Metadata:
    title           : av2_0
  Duration: N/A, start: 0.040000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 720x576, 25 fps, 25 tbr, 90k tbn, 180k tbc
Output #0, rtsp, to 'rtsp://localhost:9000/camera1':
  Metadata:
    title           : av2_0
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 720x576, q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[rtsp @ 0x5e0a3706a960] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[rtsp @ 0x5e0a3706a960] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
frame= 2568 fps= 25 q=-1.0 Lsize=N/A time=00:01:42.64 bitrate=N/A speed=   1x

Can I remove this delay for ip camera? Is it necessary to change some params for ip camera or add extra arguments for ffmpeg command? I tried to minimize delay, but it didn't have any effect.

I have solved the problem. Removing "-re" option in ffmpeg pipeline sets delay to 0.
My final pipeline:
ffmpeg -rtsp_transport udp -i rtsp://192.168.3.107:554/av2_0 -codec copy -f rtsp rtsp://localhost:9000/camera1


text:
ffmpeg - overlay position at the end of the video is wrong
I am trying to move a shape (an image) over a background image to each corner. Here is the video example I generated with ffmpeg.
As you can see from the example, at the end of the video the shape is not positioned on the top right corner as it should be:

Here is the ffmpeg command I use:
ffmpeg -i background.jpeg -loop 1 -i logo.png -filter_complex "[0]scale=300:250[s0];[1]format=gbrap[s4];[s0][s4]overlay=x=if(between(t\\,0.0\\,0.8)\\,lerp(0.0\\,240.0\\,round((t-0.0)/0.8*100)/100)\\,if(between(t\\,0.8\\,2.2)\\,lerp(240.0\\,0.0\\,round((t-0.8)/1.4*100)/100)\\,if(between(t\\,2.2\\,3.0)\\,lerp(0.0\\,240.0\\,round((t-2.2)/0.8*100)/100)))):y=if(between(t\\,0.0\\,0.8)\\,lerp(0.0\\,200.0\\,round((t-0.0)/0.8*100)/100)\\,if(between(t\\,0.8\\,2.2)\\,lerp(200.0\\,200.0\\,round((t-0.8)/1.4*100)/100)\\,if(between(t\\,2.2\\,3.0)\\,lerp(200.0\\,0.0\\,round((t-2.2)/0.8*100)/100))))[s5]" -map "[s5]" -c:v libx264 -crf 0 -pix_fmt yuv420p -t 3.0 video.mp4

video is 300x250
logo.png is the blue shape (width=60px, height=50px)
the coordinates are set as follow: 0s: x=0,y=0 / 0.8s: x=240px,y=200px / 2.2s: x=0,y=200px / 3s: x=240px,y=0

I am using lerp to retrieve the right position depending on the current time t.
It seems like the last frames are not played.
I tried to:

set a framerate (30 & 100) to the logo.png input,
round the value given by lerp,
use setpts to define a number of frames.

But I do not understand the problem I am facing, could you help me?
UPDATE 1: If I extend the video duration to 4s, the shape position at 3s is the right one
UPDATE 2: I added -framerate 100 as option the logo.png input and fps=24 just after the scale filter. The problem is still present. I extracted every frame as png (as expected there are 72 frames) & 17 were generated between 2.2s & 3s.
UPDATE 3: The problem is resolved by settings a fps of 100. However the generated video file is too big. Is there any other way to get it to work at 24fps?

Since your total duration is 3 seconds, there is no frame at 3 seconds. The last frame is at 3 seconds - 1/framerate seconds.
The default framerate of image inputs is 25, so you should interpolate to the final position at 2.96 seconds.
i.e.
if(between(t\\,2.2\\,3.0)\\, lerp(200.0\\,0.0\\,round((t-2.2)/0.76*100)/100))

Thank you, that's working. Can you tell me why "The last frame is at 3 seconds - 1/framerate seconds"? (Is this a FFmpeg limitation?)

No. Each frame has a presentation time and a duration. For 25 fps videos, duration is 1/25 = 40ms. If there was a frame at 3s, then total video duration would be 3s40ms.

oh that makes perfect sense! thank you

source: Stack Exchange

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

sort: votes


text:
FFMPEG - Partial fade effects
I'm having difficulty achieving the desired fading effect for the logo in my video using ffmpeg.
I would like to make a 3-second video (here is an example) with:

A background image (300x350 jpeg file)
A logo overlaid in the center of the background image (100x40 png file)
And I would like the logo to fade in and out with the following opacity timeline:

At 0 second, the opacity is 0.5
At 1 second, the opacity is 1
At 2 second, the opacity is 0.2
At 3 second, the opacity is 1.

As the fade filter does not support any target or source value (it is a full fade from 0 to 1 or the reverse), I cannot use it.
For example:
Here is how to add an overlay with a fully fade IN (between 0 to 1s) and a fully fade OUT (from 2s to 3s):
ffmpeg -loop 1 -i background.jpeg -framerate 30 -loop 1 -i logo.png -filter_complex "[0]format=rgba,scale=300:250[o];[1]format=rgba,fade=in:st=0:d=1:alpha=1,fade=out:st=2:d=1:alpha=1[i];[o][i]overlay=x=(W-w)/2:(H-h)/2" -c:v libx264 -t 3 output.mp4

I also tried the geq filter with a lerp operation but it is quite difficult to write (and the result I got is wrong, I may have a problem with the lerp operator):
ffmpeg -i background.jpeg -framerate 30 -loop 1 -i logo.png -filter_complex "[0]scale=300:250[o];[1]format=yuva420p,geq=a='if(lt(T\,1)\,lerp(0\,1\,T)*0.5*alpha(X\,Y)\,if(lt(T\,2)\,lerp(1\,2\,T)*0.2*-alpha(X\,Y),lerp(2\,3\,T)*alpha(X\,Y)))':r='r(X\,Y)'[i];[o][i]overlay=x=(W-w)/2:(H-h)/2" -c:v libx264 -t 3 -pix_fmt yuv420p -crf 0 output.mp4

I also found that scripting filters (https://trac.ffmpeg.org/wiki/FilteringGuide#ScriptingFilters) are also available for the geq filter but I haven't found a way to use it for that specific use case.
What would be the more appropriate approach to handle that kind of effect?

try geq=a='if(lt(T,1),lerp(0.5,1,T)*alpha(X,Y),if(lt(T,2),lerp(1,0.2,T-1)*alpha(X,Y),lerp(0.2,1,T-2)*alpha(X,Y)))':r='r(X,Y)'

Thank you, that works. However, do you know of a simpler way to achieve this effect? Maybe the scripting filter way?

u can try: ...[1]format=gbrap, split=3[s1][s2][s3]; [s1]geq=a='lerp(0.5,1,T)*alpha(X,Y)':r='r(X,Y)'[v1]; [s2]geq=a='lerp(1,0.2,T)*alpha(X,Y)':r='r(X,Y)'[v2]; [s3]geq=a='lerp(0.2,1,T)*alpha(X,Y)':r='r(X,Y)'[v3]; [v1][v2][v3]concat=n=3:v=1:a=0[i];... and insert -t 1 before logo.png

authors: Баяр Гончикжапов

all_licenses: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/


text:
verify lossless in H.264
i've transcoded an interlaced JPEG2000 video file to H.264 using ffmpeg.  following the guidelines, i constructed the following command line:
ffmpeg -hide_banner -i $input \
        -vf tinterlace=0 \
        -c:v libx264 -crf 0 -qp 0 -preset veryslow \
        -x264-params interlaced=1 -flags +ildct+ilme \
        -profile:v high444 \
        -map 0 -c:a aac -c:d copy -c:s copy -c:t copy -y $output

here is part of the mediainfo report of the resulting video (non-video tracks excluded for brevity):
General
Complete name                            : output.mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 263 MiB
Duration                                 : 1 min 0 s
Overall bit rate                         : 36.8 Mb/s
Writing application                      : Lavf60.3.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:2:2@L4
Format settings                          : CABAC / 16 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 16 frames
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 1 min 0 s
Bit rate                                 : 36.5 Mb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Bit depth                                : 10 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 3.523
Stream size                              : 261 MiB (99%)
Writing library                          : x264 core 164 r3106 eaa68fa
Encoding settings                        : cabac=1 / ref=16 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=0 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=0.0 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Color range                              : Full

my question is: is there a way to tell if the resulting output file truly is lossless?  (there is no output line that says 'lossless' in the ffmpeg output nor in the mediainfo report on the file.)
[NB - even though it might not be common modern practice to create interlaced video, the output video needs to preserve interlacing because it's part of The Requirements®.]

Just do this:
ffmpeg -i $input -i $output -lavfi ssim -f null -

This will compare the two videos using SSIM. If it's lossless then you will see:
SSIM Y:1.000000 (inf) U:1.000000 (inf) V:1.000000 (inf) All:1.000000 (inf)

You don't need both -qp 0 and -crf 0 btw. The latter is lossless if the input is 8-bit. The former always is. Also you don't need to set a profile manually. FFmpeg will pick the right profile for you.


text:
Why does setting the bitrate with ffmpeg has no affect on the filesize?
I am trying to lower the bitrate of a video using ffmpeg to 1000 bits/s and 500 bits/s using the commands
ffmpeg -i __temp__.mp4  -vf "scale=-2:360" -b:v 1000 out_1000.ts
ffmpeg -i __temp__.mp4  -vf "scale=-2:360" -b:v 500 out_500.ts

but the output files seem to have the same size:
-rw-rw-r-- 1 alex alex 8160140 Nov  2 13:11 out_1000.ts
-rw-rw-r-- 1 alex alex 8160140 Nov  2 13:11 out_500.ts

Size of original video: 8344903
Framerate: 15 fps

What is going on?

1000 bits/s is absurdly low - maybe not even enough to form all the syntax elements of a coded bitstream. Maybe you meant 1000k.

Yes I know it is low. I meant 1000, not 1000k.

The default encoder for mpegts is mpeg2video (producing a H.262 bitstream). This is an ancient encoder for an ancient codec and it is not capable of supporting a 1k or 0.5k video. Even modern codecs with significantly better compression ratios cannot get this low at 15fps. MPEG2 is also not optimized for low bitrate in the first place.
The output files have the same size because it is overshooting your target bitrate. If you want to set a hard limit then you have to add -maxrate and -bufsize in addition to -b:v. But you'll be disappointed. The encoder isn't able to do anything with such a low bitrate. If you force a hard upper limit you will see:
cropbox:~$ ffmpeg -v error -f lavfi -i "testsrc2=s=640x360:r=15" -c:v mpeg2video -b:v 500 -maxrate 500 -bufsize 1000 -f null -
[mpeg2video @ 0x5b52cea9ec80] rc buffer underflow
[mpeg2video @ 0x5b52cea9ec80] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x5b52cea9ec80] rc buffer underflow
[mpeg2video @ 0x5b52cea9ec80] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x5b52cea9ec80] rc buffer underflow
[mpeg2video @ 0x5b52cea9ec80] max bitrate possibly too small or try trellis with large lmax or increase qmax
[mpeg2video @ 0x5b52cea9ec80] rc buffer underflow
[mpeg2video @ 0x5b52cea9ec80] max bitrate possibly too small or try trellis with large lmax or increase qmax
...

Encoders aren't able to make anything useful from such a low bitrate, even modern ones. You can see the difference yourself by running these test commands (top left to bottom right: original 640x360@15 test sample, AVC, HEVC, AV1, all with a 1 second open GOP and 40kbps CBR):
ffmpeg -f lavfi -i "testsrc2=640x360:r=15:d=30" -c:v libx264 -x264-params "vbv-bufsize=40:vbv-maxrate=40:open-gop=1:keyint=15" avc.mkv
ffmpeg -f lavfi -i "testsrc2=640x360:r=15:d=30" -c:v libx265 -x265-params "strict-cbr=1:bitrate=40:vbv-bufsize=40:keyint=15" hevc.mkv
ffmpeg -f lavfi -i "testsrc2=640x360:r=15:d=30" -c:v libsvtav1 -svtav1-params "rc=2:tbr=40:pred-struct=1:irefresh-type=1:keyint=15" av1.mkv
ffplay -f lavfi -i "testsrc2=640x360:r=15:d=30,pad=iw+4:ih+4[orig];movie=avc.mkv,pad=iw+4:ih+4[avc];movie=hevc.mkv,pad=iw+4:ih+4[hevc];movie=av1.mkv,pad=iw+4:ih+4[av1];[orig][avc][hevc][av1]xstack=4:grid=2x2,fps=15[out0]"

If this looks so horrible even with advanced modern formats, how could an ancient format like MPEG2 possibly operate at 80 times lower bitrate than any of these were using?
Even when encoding a 640x360@15 pure black image, none of those encoders were able to get below 2kbps (and mpeg2video requires 62kbps for that!). As Gyan said in a comment, the overhead of the syntax elements alone is probably higher than 1kbps.


text:
Creating a Video from Pano2VR results in corrupted frames
Has anyone been able to use Pano2VR Pro to generate videos of their tours using the Animation Output option? Pano2VR uses ffmpeg to generate video frames.
I have attempted to use this feature numerous times over the years and always get bad results. The video frames it generates for the video has bits and pieces of 2-3 photos overlapped in the frame. I am attaching a sample.

This has happen on 3 different computers so I am fairly confident it isn't hardware related. I can create html type tours with no problem. It only happens if I try to create a set of images to make a video. It happens if I enable the Generate Video File or leave it disabled. Note that I have the ffmpeg installed.
In this screenshot of the Pano2VR screen, the photo that I am using to attempt to create the video from is a 2.85 Giga-pixel (37,788 x 75,576 pixels) image.

Currently i am using Pano2VR on a Windows 11 OS to create virtual tours of my Gigapixel photography.  I am running a 13th Generation Raptor Lake Intel(R) Core(TM) i9-13900K 24 Core, 32 thread CPU, ASUS ProART Z790 Creator Motherboard is populated with 128 Gigabytes of DDR5 memory,  ASUS GEFORCE RTX 4070 Ti GPU video card.
Any suggestions anyone might have would be most appreciated. I would really like to be able to create videos from Pan02VR.

Without Generate Video File, does it create an image sequence? If so, view the images externally and check whether they look ok.

Yes, it creates an image sequence with Generate Video File turned OFF. The corrupted image file happens regardless of if I enable the Generate Video File or leave it disabled. The first image on my initial post is one of the created images.

That sounds like a Pano2VR issue as ffmpeg is only used for the Generate Video File option.


text:
Most efficiant way to create a video with series of images faded in and out with ffmpeg
I am creating a video with audio that consists of 25 images that fade into each other for the duration of an audio file. There is also some text that will appear on the finished video.
What I currently have works but it is terrible slow. On an AMD 3500x using all 6 cores it can only get between .9 and 1.1 frames per second using libx264 to encode a mp4 file. Using h264_nvenc I can only get slightly better at 1.1 to 1.3 frames per second. I am going to be making 100's of these videos so would really like to do it as efficiently as possible.
It seems to be the fades that cause the problem as I can encode at 7 to 12 fps if it just jumps from one image to the other without the fade. And it seems cuda doesn't have anything that can speed up the image processing.
Here is what I have that works but I'm sure it can be made more compact. I have tried different things like setting a time -t to the input of each image but that didn't help. I have tried doing just a fade in over the previous image instead of a fade out then the next image fading in. My thinking was half as many filters would make it faster but it didn't change the speed of encoding at all.
#!/bin/bash

full_path="/Videos"
a_path=$full_path"/"$1"/"$2".wav"

images=25

duration_function() {
    ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 $a_path
        return 100
    }

duration="$(duration_function "$a_path")"

duration=${duration%.*}
minutes=$((duration / 60))
seconds=$((duration / 60))
seconds=$(printf "%02d" $seconds)
fade=$((duration / images))

intro1=1
intro2=$((fade+2))
intro3=$((intro2+fade))
intro4=$((intro3+fade))
intro5=$((intro4+fade))
intro6=$((intro5+fade))
intro7=$((intro6+fade))
intro8=$((intro7+fade))
intro9=$((intro8+fade))
intro10=$((intro9+fade))
intro11=$((intro10+fade))
intro12=$((intro11+fade))
intro13=$((intro12+fade))
intro14=$((intro13+fade))
intro15=$((intro14+fade))
intro16=$((intro15+fade))
intro17=$((intro16+fade))
intro18=$((intro17+fade))
intro19=$((intro18+fade))
intro20=$((intro19+fade))
intro21=$((intro20+fade))
intro22=$((intro21+fade))
intro23=$((intro22+fade))
intro24=$((intro23+fade))
intro25=$((intro24+fade))

outro1=$((fade+2))
outro2=$((outro1+fade))
outro3=$((outro2+fade))
outro4=$((outro3+fade))
outro5=$((outro4+fade))
outro6=$((outro5+fade))
outro7=$((outro6+fade))
outro8=$((outro7+fade))
outro9=$((outro8+fade))
outro10=$((outro9+fade))
outro11=$((outro10+fade))
outro12=$((outro11+fade))
outro13=$((outro12+fade))
outro14=$((outro13+fade))
outro15=$((outro14+fade))
outro16=$((outro15+fade))
outro17=$((outro16+fade))
outro18=$((outro17+fade))
outro19=$((outro18+fade))
outro20=$((outro19+fade))
outro21=$((outro20+fade))
outro22=$((outro21+fade))
outro23=$((outro22+fade))
outro24=$((outro23+fade))
outro25=$((duration-1))

/usr/bin/ffmpeg -y -hide_banner -threads 6 -f lavfi -i color=c=black:s=3840x2160:r=30:d=$duration -pix_fmt rgba -loop 1 -i 1.png -loop 1 -i 2.png -loop 1 -i 3.png -loop 1 -i 4.png -loop 1 -i 5.png -loop 1 -i 6.png -loop 1 -i 7.png -loop 1 -i 8.png -loop 1 -i 9.png -loop 1 -i 10.png -loop 1 -i 11.png -loop 1 -i 12.png -loop 1 -i 13.png -loop 1 -i 14.png -loop 1 -i 15.png -loop 1 -i 16.png -loop 1 -i 17.png -loop 1 -i 18.png -loop 1 -i 19.png -loop 1 -i 20.png -loop 1 -i 21.png -loop 1 -i 22.png -loop 1 -i 23.png -loop 1 -i 24.png -loop 1 -i 25.png -i $a_path -filter_complex \
"[1]format=rgba,fade=in:st=$intro1:d=0.5:alpha=1,fade=out:st=$outro1:d=0.5:alpha=1[1a];\
[0:v][1a]overlay=x=0:y=0[out];\
[2]format=rgba,fade=in:st=$intro3:d=0.5:alpha=1,fade=out:st=$outro3:d=0.5:alpha=1[2a];\
[out][2a]overlay=x=0:y=0[out];\
[3]format=rgba,fade=in:st=$intro3:d=0.5:alpha=1,fade=out:st=$outro3:d=0.5:alpha=1[3a];\
[out][3a]overlay=x=0:y=0[out];\
[4]format=rgba,fade=in:st=$intro4:d=0.5:alpha=1,fade=out:st=$outro4:d=0.5:alpha=1[4a];\
[out][4a]overlay=x=0:y=0[out];\
[5]format=rgba,fade=in:st=$intro5:d=0.5:alpha=1,fade=out:st=$outro5:d=0.5:alpha=1[5a];\
[out][5a]overlay=x=0:y=0[out];\
[6]format=rgba,fade=in:st=$intro6:d=0.5:alpha=1,fade=out:st=$outro6:d=0.5:alpha=1[6a];\
[out][6a]overlay=x=0:y=0[out];\
[7]format=rgba,fade=in:st=$intro7:d=0.5:alpha=1,fade=out:st=$outro7:d=0.5:alpha=1[7a];\
[out][7a]overlay=x=0:y=0[out];\
[8]format=rgba,fade=in:st=$intro8:d=0.5:alpha=1,fade=out:st=$outro8:d=0.5:alpha=1[8a];\
[out][8a]overlay=x=0:y=0[out];\
[9]format=rgba,fade=in:st=$intro9:d=0.5:alpha=1,fade=out:st=$outro9:d=0.5:alpha=1[9a];\
[out][9a]overlay=x=0:y=0[out];\
[10]format=rgba,fade=in:st=$intro10:d=0.5:alpha=1,fade=out:st=$outro10:d=0.5:alpha=1[10a];\
[out][10a]overlay=x=0:y=0[out];\
[11]format=rgba,fade=in:st=$intro11:d=0.5:alpha=1,fade=out:st=$outro11:d=0.5:alpha=1[11a];\
[out][11a]overlay=x=0:y=0[out];\
[12]format=rgba,fade=in:st=$intro12:d=0.5:alpha=1,fade=out:st=$outro12:d=0.5:alpha=1[12a];\
[out][12a]overlay=x=0:y=0[out];\
[13]format=rgba,fade=in:st=$intro13:d=0.5:alpha=1,fade=out:st=$outro13:d=0.5:alpha=1[13a];\
[out][13a]overlay=x=0:y=0[out];\
[14]format=rgba,fade=in:st=$intro14:d=0.5:alpha=1,fade=out:st=$outro14:d=0.5:alpha=1[14a];\
[out][14a]overlay=x=0:y=0[out];\
[15]format=rgba,fade=in:st=$intro15:d=0.5:alpha=1,fade=out:st=$outro15:d=0.5:alpha=1[15a];\
[out][15a]overlay=x=0:y=0[out];\
[16]format=rgba,fade=in:st=$intro16:d=0.5:alpha=1,fade=out:st=$outro16:d=0.5:alpha=1[16a];\
[out][16a]overlay=x=0:y=0[out];\
[17]format=rgba,fade=in:st=$intro17:d=0.5:alpha=1,fade=out:st=$outro17:d=0.5:alpha=1[17a];\
[out][17a]overlay=x=0:y=0[out];\
[18]format=rgba,fade=in:st=$intro18:d=0.5:alpha=1,fade=out:st=$outro18:d=0.5:alpha=1[18a];\
[out][18a]overlay=x=0:y=0[out];\
[19]format=rgba,fade=in:st=$intro19:d=0.5:alpha=1,fade=out:st=$outro19:d=0.5:alpha=1[19a];\
[out][19a]overlay=x=0:y=0[out];\
[20]format=rgba,fade=in:st=$intro20:d=0.5:alpha=1,fade=out:st=$outro20:d=0.5:alpha=1[20a];\
[out][20a]overlay=x=0:y=0[out];\
[21]format=rgba,fade=in:st=$intro21:d=0.5:alpha=1,fade=out:st=$outro21:d=0.5:alpha=1[21a];\
[out][21a]overlay=x=0:y=0[out];\
[22]format=rgba,fade=in:st=$intro22:d=0.5:alpha=1,fade=out:st=$outro22:d=0.5:alpha=1[22a];\
[out][22a]overlay=x=0:y=0[out];\
[23]format=rgba,fade=in:st=$intro23:d=0.5:alpha=1,fade=out:st=$outro23:d=0.5:alpha=1[23a];\
[out][23a]overlay=x=0:y=0[out];\
[24]format=rgba,fade=in:st=$intro24:d=0.5:alpha=1,fade=out:st=$outro24:d=0.5:alpha=1[24a];\
[out][24a]overlay=x=0:y=0[out];\
[25]format=rgba,fade=in:st=$intro25:d=0.5:alpha=1,fade=out:st=$outro25:d=0.5:alpha=1[25a];\
[out][25a]overlay=x=(main_w-overlay_w):y=(main_h-overlay_h)[out];[out]drawtext=fontfile=/font.ttf:text='Text':fontcolor=white:fontsize=50:box=1:boxcolor=black@.4:boxborderw=9:x=300:y=1810,\
drawtext=fontfile=/font.ttf:text='\:':fontcolor=red:fontsize=50:box=1:boxcolor=black@0:boxborderw=5:x=762:y=1813:enable='lt(mod(t,1),.5)',\
drawtext=fontfile=/font.ttf:fontsize=50:fontcolor=white:text='Elapsed Time  %{pts\:gmtime\:0\:%M\\\\\:%S} / $minutes\:$seconds':box=1:boxcolor=black@.4:boxborderw=9:x=300:y=1860,
drawtext=fontfile=/font.ttf:fontsize=50:fontcolor=red:text='\:':box=1:boxcolor=black@.4:boxborderw=9:x=823:y=1865:enable='lt(mod(t,1),.4)',
drawtext=fontfile=/font.ttf:text='Text':y=1712:x=(if(lt(10*n/(300+tw)\,1)\,mod(10*n\,300+tw)-tw\,300)):fontcolor=white:fontsize=85:enable='between(t,0,$((duration-1)))',\
drawtext=fontfile=/font.ttf:text='Text':y=1711:x=(if(lt(10*n/(300+tw)\,1)\,mod(10*n\,300+tw)-tw\,300)):fontcolor=red:fontsize=85:enable='between(t,0,$((duration-1)))'[out]"\
 -map "[out]" -map 26:a -c:v libx264 -crf 18  -acodec aac video.mp4

So I'm not sure if setting a color format would help or what I can do. I know it is 4k and will be slow but for most of the video all it really needs to do is display a static picture but it seem like because there is an alpha channel it slows it way doing even it the transparency for most of the frames is at 0%.
Thank you to anyone who can help me make this more efficient!
Thanks to the helpful links provided I ended up with this and it runs at over 20 fps and looks great. I wish I could get the cuda HW acceleration to work but I couldn't figure out the hwupload hwdoanload and told me the pixel formats were incompatable but I am very happy with this!
#!/bin/bash

full_path="/Videos"
a_path=$full_path"/"$1"/"$2".wav"
font="/font.ttf"
images=25

duration_function() {
    ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 $a_path
    return 100
}

duration="$(duration_function "$a_path")"
duration=${duration%.*}

minutes=$((duration / 60))
seconds=$((duration % 60))

seconds=$(printf "%02d" $seconds)
fade=$((duration / images))

intro1=1
intro2=$((intro1+fade))
intro3=$((intro2+fade))
intro4=$((intro3+fade))
intro5=$((intro4+fade))
intro6=$((intro5+fade))
intro7=$((intro6+fade))
intro8=$((intro7+fade))
intro9=$((intro8+fade))
intro10=$((intro9+fade))
intro11=$((intro10+fade))
intro12=$((intro11+fade))
intro13=$((intro12+fade))
intro14=$((intro13+fade))
intro15=$((intro14+fade))
intro16=$((intro15+fade))
intro17=$((intro16+fade))
intro18=$((intro17+fade))
intro19=$((intro18+fade))
intro20=$((intro19+fade))
intro21=$((intro20+fade))
intro22=$((intro21+fade))
intro23=$((intro22+fade))
intro24=$((intro23+fade))
intro25=$((intro24+fade))

ffmpeg -y -hide_banner -threads 6 -f lavfi -i color=c=black:s=3840x2160:r=25:d=$duration -loop 1 -i 1.png -loop 1 -i 2.png -loop 1 -i 3.png -loop 1 -i 4.png -loop 1 -i 5.png -loop 1 -i 6.png -loop 1 -i 7.png -loop 1 -i 8.png -loop 1 -i 9.png -loop 1 -i 10.png -loop 1 -i 11.png -loop 1 -i 12.png -loop 1 -i 13.png -loop 1 -i 14.png -loop 1 -i 15.png -loop 1 -i 16.png -loop 1 -i 17.png -loop 1 -i 18.png -loop 1 -i 19.png -loop 1 -i 20.png -loop 1 -i 21.png -loop 1 -i 22.png -loop 1 -i 23.png -loop 1 -i 24.png -loop 1 -i 25.png -i $a_path -filter_complex "\
[0:v][1]xfade=transition=fade:duration=2:offset=$intro1[out];\
[out][2]xfade=transition=fade:duration=2:offset=$intro2[out];\
[out][3]xfade=transition=fade:duration=2:offset=$intro3[out];\
[out][4]xfade=transition=fade:duration=2:offset=$intro4[out];\
[out][5]xfade=transition=fade:duration=2:offset=$intro5[out];\
[out][6]xfade=transition=fade:duration=2:offset=$intro6[out];\
[out][7]xfade=transition=fade:duration=2:offset=$intro7[out];\
[out][8]xfade=transition=fade:duration=2:offset=$intro8[out];\
[out][9]xfade=transition=fade:duration=2:offset=$intro9[out];\
[out][10]xfade=transition=fade:duration=2:offset=$intro10[out];\
[out][11]xfade=transition=fade:duration=2:offset=$intro11[out];\
[out][12]xfade=transition=fade:duration=2:offset=$intro12[out];\
[out][13]xfade=transition=fade:duration=2:offset=$intro13[out];\
[out][14]xfade=transition=fade:duration=2:offset=$intro14[out];\
[out][15]xfade=transition=fade:duration=2:offset=$intro15[out];\
[out][16]xfade=transition=fade:duration=2:offset=$intro16[out];\
[out][17]xfade=transition=fade:duration=2:offset=$intro17[out];\
[out][18]xfade=transition=fade:duration=2:offset=$intro18[out];\
[out][19]xfade=transition=fade:duration=2:offset=$intro19[out];\
[out][20]xfade=transition=fade:duration=2:offset=$intro20[out];\
[out][21]xfade=transition=fade:duration=2:offset=$intro21[out];\
[out][22]xfade=transition=fade:duration=2:offset=$intro22[out];\
[out][23]xfade=transition=fade:duration=2:offset=$intro23[out];\
[out][24]xfade=transition=fade:duration=2:offset=$intro24[out];\
[out][25]xfade=transition=fade:duration=2:offset=$intro25[out];\
[out]drawtext=fontfile=$font:text='Text':fontcolor=white:fontsize=50:box=1:boxcolor=black@.4:boxborderw=9:x=300:y=1810,\
drawtext=fontfile=$font:text='\:':fontcolor=red:fontsize=50:box=1:x=762:y=1813:enable='lt(mod(t,1),.5)',\
drawtext=fontfile=$font:fontsize=50:fontcolor=white:text='Elapsed Time  %{pts\:gmtime\:0\:%M\\\\\:%S} / $minutes\:$seconds':box=1:boxcolor=black@.4:boxborderw=9:x=300:y=1860,
drawtext=fontfile=$font:fontsize=50:fontcolor=red:text='\:':x=823:y=1865:enable='lt(mod(t,1),.4)',
drawtext=fontfile=$font:text='Text':y=1712:x=(if(lt(10*n/(300+tw)\,1)\,mod(10*n\,300+tw)-tw\,300)):fontcolor=white:fontsize=85:enable='between(t,0,$((duration-1)))',\
drawtext=fontfile=$font:text='Text':y=1711:x=(if(lt(10*n/(300+tw)\,1)\,mod(10*n\,300+tw)-tw\,300)):fontcolor=red:fontsize=85:enable='between(t,0,$((duration-1)))'[out]"\
 -map "[out]" -map 26:a -shortest -c:v hevc_nvenc -rc vbr -cq 24 -qmin 24 -qmax 24 -profile:v main10 -b:v 0K -c:a aac -map 0 video.mp4

Try this. Determine the length of the audio and calculate the fade in-out length per image. Then generate 25 individual video files from each of the 25 images with the fade in-out effect. Then use the ffmpeg concat demuxer to merge all the 25 videos files into one video (this will join the videos together without re-encoding it). Finally add the audio to the final video.

See also - 3 Methods you need to know about FFmpeg transition animation and experiment which is the fastest.

Is it that ffmpeg has to calculate all the images even if only one of them is visible? I made a test with just two images thinking that was the case and even with only two images it was the same slow speed. Or would I have to make videos that were just the 1 second transitions and the 15 or so odd seconds where there was no transition would be a separate video file too. So I would need 50 videos. I could probably script that so it wouldn't be so bad. I'm not sure how to deal with the text. Some is scrolling and it takes the cues from the total length of the video & there is a elapsed counter.

Thank you for the links! They pointed me to xfade and now I am getting over 20 fps second the quality looks great!


text:
Retrieve and decode V-Chip data from an aired digital TV stream
I'm trying to retrieve and decode V-chip data from my TV streams (NTSC/ATSC digital streams). Apparently these data are embedded somewhere in the stream itself, but I have no clue how they are embedded inside, if they are inside visual frames, or a different hidden channel.
How are they embedded in the stream? And how can I retireve these data? Is there a software tool to get those signals? I need to do this programmatically in code.

V-Chip is encoded in the Extended Data Services (XDS) that is part EIA-608 captions. How it is actually transported in/with the video depends the signal type and medium.

Analog: Encoded with the captions on line 21 field 2.
SD SDI: Probably also on line 21 field 2.
HD SDI: On line 9 of the HANC encoded as per SMPTE ST 334.
ATSC over the air: In the DTVCC transport channel.
In a file: It depends on the wrapper.

It can be decoded in software. However, I don't know of any tools that aren't part of something larger, like transcoding, that can do it.

Looks like tools like mediaInfo can show it from a .ts file.


text:
Call FFmpeg filter from source code (C API)
I'm trying to find out how to call a FFmpeg filter (freezedetect) from within FFmpeg's source code. I see the filter's file is under libavfilter/vf_freezedetect.c, but I don't know how to use that filter inside the source code. How can I do that? And what I need to pass to the function?
My FFmpeg command's high-level API is this:
ffmpeg -i freeze.mp4 -vf "freezedetect=n=-60dB:d=2" -map 0:v:0 -f null -

And what I would need to get the resulting outputs directly after calling a function given its input arguments:
lavfi.freezedetect.freeze_start: 5.005
lavfi.freezedetect.freeze_duration: 2.03537
lavfi.freezedetect.freeze_end: 7.04037
lavfi.freezedetect.freeze_start: 13.0464
lavfi.freezedetect.freeze_duration: 3.03637
lavfi.freezedetect.freeze_end: 16.0827

Just for some context, are you already familiar with C programming and the ffmpeg APIs? If not I suggest you start with the dranger.com tutorial on reading and parsing first. If you are comfortable with that, you can ask this question on stackoverflow and I can help with creating pads and filters.

I am familiar with C, but not FFmpeg C API. Knowing how to call this filter from C API would help.

Also see doc/examples in FFmpeg source code, such as filtering_video.c.

Sure. I hope the answer below can be detailed enough to implement. I'm not super C guy.

@llogan Is filter API sequential, or multi-threaded? I mean, when calling filters, do they run sequential, not on a separate thread?

@TinaJ I'm not sure, but you can view the output of ffmpeg -filters to see which filters support slice threading.

You can't just call a filter, You must instantiate a filter graph.
First you need to create a buffer and buffersink via:
AVFilterContext* source = 0;
AVFilterContext* sink = 0;
AVFilterGraph *graph = avfilter_graph_alloc()
avfilter_graph_create_filter(&source, avfilter_get_by_name("buffer"), nullptr, args, nullptr, graph);

AVBufferSinkParams* buffersink_params = av_buffersink_params_alloc();
buffersink_params->pixel_fmts = /*Your pixel format HERE*/;

avfilter_graph_create_filter(&sink, avfilter_get_by_name("buffersink"), nullptr, nullptr, buffersink_params, graph);

You can then create your filter via something like this
snprintf(args, sizeof(args), "n=-60dB:d=2");
avfilter_graph_create_filter(&freezedetect, avfilter_get_by_name("freezedetect"), nullptr, args, nullptr, graph);

Then link it all together:
avfilter_link(source, 0, freezedetect, 0);
avfilter_link(freezedetect, 0, sink, 0);
avfilter_graph_config(graph, 0);

Finally you can send frames to the filter via
av_buffersrc_write_frame(source, frame);

And get the result with
 av_buffersink_get_frame(sink, frame);

Don't forget to free it all at the end

umm looks very promising! And at the same time not easy though. Thanks!

Yeah, Filters are pretty complicated.

Is filter API sequential, or multi-threaded? I mean, when calling filters, do they run sequential, not on a separate thread?


text:
Issues in trimming a video with FFmpeg
I have a .webm video captured through a web browser. Using ffmpeg, I want to trim it into its first 30s without re-encoding and quality degradation. But I have issues doing it and ffmpeg complains about it.
What is the problem? How can I address it?
ffmpeg -i video.webm -to 30 -c copy out.webm

Error:
[webm @ 0000014ea3249a00] Only VP8 or VP9 or AV1 video and Vorbis or Opus audio and WebVTT subtitles are supported for WebM.
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:1 --

WebRTC saves even H264 to webm which is not compatible. Save to mkv.

Would it work to save to .mp4? Any issues? It works for me though.

Depends on the codecs.

AVC video and Opus audio.

AVC is fine  Opus is not commonly accepted in MP4. Transcode audio to AAC.

I renamed it as .mp4 and Opus looks fine.


text:
Digital TV SCTE 35 signal
There is a standard for Cable TV called SCTE 35 that can signal ad insertion. I need to play around with this signal.
I have a ATSC receiver now, and I can record TV into .TS file format. How can I check if a channel includes SCTE 35 signals?
UPDATE: Upon checking, unfortunately none of my current 4 channels include SCTE-35 signal (US ATSC receiver: Bounce, Escape, LAFF, Create, KUVN). Do we have any digital TV channels in the US that have that signal?

What are you trying to accomplish and what software are you using?

Actually I used TVHeadend which also shows if a channel has SCTE-35 (0x52 PID). DVB Inspector as said below also does that. Unfortunately none of my current channels include that. Do we even any one channel having that signal in the US?

I asked about what you are trying to accomplish because I somewhat expect that they would be stripped before going to consumers.  The purpose is to allow for a local broadcaster to insert their local advertisements in to the stream automatically.  There is no reason that information would need to be sent to the end consumers and I don't see why it would be forwarded in to the signals being sent out to consumers.

You are right. The provider probably removes it at the time we receive it. I was trying to use that signal to classify videos and ads.

To determine if a channel has SCTE-35, you will need to have the Program Map Table (PMT) recorded. From the PMT, you should be able to find the Packet Identifiers (PIDs) of all of the streams associated with the channel you are interested in. There will be a PID for the video and at least one audio PID. If SCTE-35 is present, there will be at least one PID with a stream type of 0x52.
Keep in mind, the PMT and the SCTE-35 PID will either need to be in the clear or you will need to be able to decrypt the conditional access.
The easiest way to check all of this is with a transport stream analyzer. Most of the freely available ones, like DVB Inspector, will show you the PMT. However, you will probably need a commercial analyzer to get it to display something more than just the raw SCTE-35 bytes.

Thanks. So if I record a channel into a .ts file, the dvb inspector can say if it includes the flag? For now i only need to know if it has scte 35. Not really the value.

I haven't personally used DVB Inspector, but it came recommended by a reliable source. In theory it should be able to at least show you if the PID exists. It may not show more than stream type in hex and the raw content of the PID.

Here is a screenshot of the table: https://i.imgur.com/liPBoNr.png I tested for a few channels (Bounce, Escape, Laff), and none had PID of 0x52.

Do you know of any channels having that? Where can I find a list of ATSC channels which have SCTE 35?

If you are are capturing the transport stream off an offer the air channel of a cable system, they are likely grooming out the SCTE-35. You would need to look at the provider feed to the station or MPVD. Most major cable networks should have DPI triggers at that point.

You are right. The provider probably removes it at the time we receive it. I need to find another way. My whole aim is to just classify ad contents and video contents.

Try threefive.
threefive returns all SCTE-35 data (Splice Info Section, Splice Command, and Splice Descriptors) in the proper format according to the 2019 specification.
https://github.com/futzu/SCTE35-threefive

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/3.0/


text:
Objective video quality assessment
I'm trying to develop a tool that given a video (without a reference video), it can measure the quality. Basically given some thresholds, it assigns them a rating (I just need 4 classes 1-4). Since we don't have the original reference videos, I'm not looking for approaches using SSIM or PSNR or such. Possible options are using machine learning approaches.
Is there any available tool or package for that? If not, how can I implement one?

If I understand your question correctly this https://github.com/Netflix/vmaf could help

@Marware But that one is still reference-based. Isn't it?

reference_path and distorted_path are the paths to the reference and distorted video files in ./run_vmaf format width height reference_path distorted_path [--out-fmt output_format]

What you are looking for is called “no-reference quality assessment”. Google can give you specific examples.

Couldn't find an available tool for that, except some research papers.

Which answers your question as reading those research papers will allow you to implement your own tool. @TinaJ ¯\(ツ)/¯ +1

Haha I'm no more a phd student to spend time on that! Would prefer a ready-to-use tools.


text:
How to convert .scc closed caption subtitles into human readable text
I'm trying to find a tool or library (possibly an open-source tool) to parse or convert .scc closed caption subtitle files to a human readable text, say .srt or xml or any text formats. Is there a strong tool or library to do that?
Previously, I was trying to use PyCaption library in Python. But it fails to parse certain .scc files.

Subtitle Edit can convert to/from .scc, .srt, and many other formats. It is GPLv3 (and possibly also LGPL licensed). It also has a BSD licensed DLL. The downside is that it is C# code.

Problem is that there is not a perfect tool. For instance, I had examples that PyCaption couldn't parse and SE parsed, and yesterday I had a sample that SE parsed, but for some reasons it concatenates multiple captions together as a single one.

SE's developer is pretty responsive, or at least was several years ago, if you can get a sample to them. It may be worth investing in MacCaption/CaptionMaker for the paid support.

yeah they are pretty responsive. No paid support.

You can also try this tool. It is scc file conversion is tested on a lot of files and is under active development. If you have any problems, report your file and it will be fixed.
php subtitles.phar input.scc output.srt


text:
Add 3D visual effects to videos
I'm trying to get some hands-on experience with 3D visual effects on videos. Similar to those AR apps, I want to add a 3D model with effects, say a dragon, to a video. Something like this famous demo:

But I don't really want to make it this fancy with camera tracking. Starting with simplest approach, probably just placing a dragon on the field (like its last scenes) with fixed camera.
What's the best way to do it? Do we have some tutorials, free or open source tools or SDK to do it for videos?

Yeah that was actually the best opening ceremony ever...
Doing something like this yourself is a hefty task, but first, here's what you would need:

A software for tracking
A software for modeling / animating
(optionally something for compositing / color grading)

Luckily, Blender (which is free) can do all those things, but has quite a steep learning-curve. Due to its popularity, you will find many tutorials and guides though. Here's step by step how I would go about something like your example:

Find footage to composite the elements on. It should be at least 1920x1080 and with as little compression as possible (it's best to record your own).
Load the footage in a tracking-program. You can use Blender, After-Effects, Nuke or PfTrack - all are viable.
After tracking, you can create a virtual camera that flies around in your scene according to the movement of the real camera. Export this or work with it from inside Blender if you're only using Blender.
Now model out and animate your 3D-Assets and place them in your scene. Since the tracked camera matches the movement of the real camera, your 3D-Assets should stay in place and look like they're pinned to the ground.
Apply your textures and shaders (and I can't stress how important this is) light your scene properly. Try to observe where the light is coming from in your original scene and place virtual lights accordingly.
Render your scene, preferably as a .png or .exr-sequence, with only your virtual Assets visible.
At last, composite the background and foreground together. You can do this in Blender as well, but programs such as After-Effects or Nuke give you more control over your look (they're not free programs though).

If your tracking looks off, make sure to delete any floating tracking-points that might have picked up false movements. Try matching the white-balance in compositing (the shift from teal to orange and purple to green). For added realism, you can create a "shadow-catcher" which will itself not be visible, but render out the shadows cast upon it. Those shadows should be included in a separate "pass", so you have your foreground, shadows and background. This gives you more control over the darkness and opacity of your shadows if you want to adjust them. And at last, if you want to go all in, you can capture an HDR environment from the place you were filming at - basically a 360° image of the surroundings. You can add this to the 3D-Software you're using to accurately display reflections on your foreground.
I hope this helped you out, have a great day!

Thanks for the detailed comments. But I don't really want to make it this fancy with camera tracking. Starting with simplest approach, probably just placing a dragon on the field (like its last scenes) with fixed camera. Do we have a tutorial for this?

Sure thing - for that approach I recommend this tutorial: https://www.youtube.com/watch?v=kzym73lhmD4
or this one:
https://www.youtube.com/watch?v=aA2XzTDad0A

Cool. Does Blender have an SDK to automatically do this?

Hmm that I don't know, you might want to ask that as a new question on here :)


text:
FFmpeg 2017 vs 2019 decoding performance improvement
I was trying to do some basic profiling on ffmpeg, following my previous question  here. I have a relatively high-bitrate .ts file (17mbps), and I have two versions of ffmpeg (2017.3 and 2019.4), and tried to run a decoding process (blackdetect filter) on both of them. I notice the 2019 version is at least 2.2 times faster. More especifically, execution of the following command takes 2.2x faster on the 2019 version:
ffmpeg -i in.ts -vf blackdetect=d=0.1:pix_th=.1 -f rawvideo -y null

Is this really expected? Is there a known, major performance improvement on ffmpeg that wasn't there in 2017 version? If that's really the case, updating my ffmpeg core should really make it run faster!
EDIT:
File is H264 encoded, but relatively higher bitrate. I actually compiled both ffmpeg versions from source code with same build options. If it is decoding improvement, that's something very interesting.
The 2019 ffmpeg command output:
ffmpeg version N-94171-g0cbdedb Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.11) 20160609
  configuration: --prefix=/home/user/usr/local --enable-shared
  libavutil      56. 30.100 / 56. 30.100
  libavcodec     58. 53.101 / 58. 53.101
  libavformat    58. 28.101 / 58. 28.101
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 56.100 /  7. 56.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

And the 2017 output:
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.11) 20160609
  configuration: --prefix=bin --bindir=bin
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

EDIT 2:
Full log for ffmpeg -i in.ts -t 100 -benchmark -an -f null -: 2019 and 2017

Added the command. Curious why we see this huge improvement.

File is H264, but high bitrate. I actually compiled both versions from source code with same build options. If it is decoding improvement, that's something I'd be interested.

@PaulB.Mahol Builds you mean just the configure outputs, or also all the make outputs?

oh ok...wait a second...

@PaulB.Mahol just added those details.

Yes. Everything the same.

@PaulB.Mahol any updates?

Total time to decode. Also, there is a speed:2.5x notation on ffmpeg when decoding.

Well, i meant if speed for 2017 was whatever, the same for 2019 was 2.2x more (I think maybe 2 vs 4.6 or something)

Share full log for ffmpeg -i in.ts -t 100 -benchmark -an -f null - with both versions.

@Gyan Added those full logs in revised question.


text:
Netflix preview video production
Netflix auto-plays a 20s, 30s video preview in the background if a video is selected or we hover the mouse pointer on them. I was trying to play with them, but couldn't understand how this preview "video production" is done.
Is there any information or insights how these previews are generated? Are they a random portion of the video, or what?
Alternatively, if you have a good suggestion of how to select a portion of video, please share.

I'm not sure, but most likely it is the trailer or a specificaly for Netflix edited Teaser

I noticed sometimes they play the trailer, but sometimes just a portion of the movie.


text:
Video clip making features with FFmpeg
I'm trying to use ffmpeg to create a video clip out of multiple videos, photos, and background music. Basically all the features that tools like Windows Movie Maker support. I would like to do things like:

merge the videos with arbitrary start/ends,
add transition effects between them,
add background music,
fade-in/fade-out/mute/stop music at certain time periods,
and more...

Can I do these with ffmpeg? Or is there any APIs in either Python or Java that I can easily use to do that?

All of these can be done - see trim, fade, volume, concat, blend, amix, filters.. etc. Support for transition effects is limited, though.

@Gyan That looks like an answer to me!  Please post one and ping me and I'll come back and upvote!  0:-)

@gyan would be great if you could post an example of each just for the sake of answer.

There are examples of each question scattered about on [so], [su], and here.


text:
ffmpeg Config to Produce H.264 Constrained Baseline Profile
It all begins with Plex and appears to end at ffmpeg as the easiest possible solution.  I love ffmpeg!  I use Plex running on Linux to stream over home network to a Roku.  I've used this for years until recently as I'll describe below.  I have a temporary workaround using Chromecast to bypass this issue.
Some videos don't play. There are two differences between the ones that play and the ones that don'that (sound but black screen).  I list these below.  Another possibility not listed is that Roku / Plex / Ubuntu have some kind of interaction (bug).  Maybe the Roku device is old and unsupported?  I listed my config way below.

ffmpeg used to generate mkv / mp4 for the ones that don't play (sound but black screen).   Handbrake and WinxDVD used to generate mkv / m4a for the ones that do play.  My ffmpeg command is:

**ffmpeg -threads 4 -f gdigrab -probesize 42M -thread_queue_size 1024 -i desktop -f dshow -thread_queue_size 1024 -i audio="CABLE Output (VB-Audio Virtual Cable)" -vf scale=720:404 -vcodec libx264 -crf 28 -t   6.0 -y  "C:/Users/daveg/Documents/Recordings\trial4_raw.mp4"**

Video file properties show codec H.264 High 4:4:4 for the ones that don't play (sound but black screen). and H.264 Constrained Baseline Profile for the ones that do play.

Does somebody know how to configure ffmpeg to generate H.264 Constrained Baseline Profile? That seems like my next logical step. I tried various crf values only to produce grainy results still with High 4:4:4 nature.
Plex 1.32.8.7639 running on Ubuntu 22.04.1
Roku 3920X s/w 12.5.0 build 4178-91
ffmpeg 4.4.2 running on Ubuntu 22.04.1

You may not actually need baseline profile, but only a compatible pixel format i.e. YUV 4:2:0 as opposed to YUV 4:4:4 that ffmpeg converts gdigrab's RGB frames to.
Add -pix_fmt yuv420p. For baseline profile, also add -profile:v baseline.

-pix_fmt yuv420 added just after libx264 did the trick.  No longer have black screen with sound for Plex on Roku.  When I type ffmpeg -pix_fmts I find a whole list of these options...research for another day.  Thanks.   I didn't try the -profile:v baseline suggestion.  Another day.


text:
Convert video from webm to mp4 using ffmpeg
I have 10 webm videos, each one has the same size ~15mb. When I run following code:
ffmpeg -i video1.webm -f mp4 video1.mp4

9 of them finish in ~10 seconds, but 1 lasts more then 3 minutes. I run ffprobe to see difference and noticed only this:
25 fps, 25 tbr, 1k tbn (default)

vs
1k tbr, 1k tbn (default)

Can someone please explain to me the difference and the best way to speed up the process to transform all 10 at nearly the same time?

That anomalous file likely has been detected (wrongly) to have a high frame rate.
Use
ffmpeg -i video1.webm -fps_mode passthrough -f mp4 video1.mp4

Your ffmpeg version should be recent i.e. at least 5.1


text:
Using FFmpeg to detect alpha channel in PR4444 file
I wanted to comment on a thread that already exists on this topic in general, but I just signed up to SO and have not earned my contribution credits yet! If anyone has advice on how to better handle that in the future, I am all ears!!
None-the-less, here is that thread:
https://stackoverflow.com/questions/69029439/a-good-way-to-detect-alpha-channel-in-a-video-using-ffmpeg-ffprobe/69030041#69030041?newreg=ba5899a955914c74a99a2e8b4ce3d12c
I tried both the answers listed in that post to detect the presence of an alpha channel in a PR4444 video file.
For Gyan's answer, I get the following pixel format from any PR4444 file (regardless if encoded with alpha or no alpha), when running part 1:
yuva444p12le
As you will see in Gyan's notes, even if a FFprobe returns an "a" in the return string (for the call for pixel format), that does not equate to the presence of an alpha channel. Hence their note for the subsequent (part 2) call...
I then plugged that into the part 2 of Gyan's answer, but that produced an empty from FFmpeg, and a message providing valid arguments to pair with grep. It seems as though -oP is not a valid arg for grep? Because it wasn't listed in the returned list of options. This is the template I used (pulled verbatim from Gyan's posted solution):
ffprobe -v 0 -show_entries pixel_format=name:flags=alpha -of compact=p=0 | grep "$PIX_FMT|" | grep -oP "(?<=alpha=)\d"
Where $PIX_FMT is to be replaced with yuva444p121e (per Gyan's instructions).
I then tried Benji's solution for part 2, but that just returns the following:
pix_fmt=yuva444p12le
Of course, this isn't any more useful than part 1...
I suspect Gyan is on the right track as he mentions part 2 should produce a boolean result for the presence of an alpha channel. However, either I am misunderstanding the syntax of his template, or something has changed in FFprobe since. There is so limited information out there for this specific task. Hoping someone with more experience and knowledge can help shed some light for me??

Your version of grep may not have PCRE enabled, which is required for P portion of -oP. In any case, if a comes right after yuv ot rgb, alpha is present.

Interesting... I encoded the file myself in After Effects and selected the RGB/Millions of Colors option in the render settings (Instead of RGBA/Millions of Colors+). Then, I brought the rendered file back into After Effects and there is no visible alpha channel present. I added a "Set Channels" effect to the footage and turned off RGB channels (left alpha channel alone). The result is a full black frame. Perhaps PR4444 just always returns yuva from the FFprobe library used for pixel format? Or perhaps PR4444 always encodes with allotment for alpha data, even if it is all zeros?

It could be the behaviour of the Adobe encoder. The ffmpeg decoder won't report alpha if the component isn't present. The presence of the channel is distinct from its contents (whether all zeros or not).


text:
kdenlive no audio in rendered video
operating system: ubuntu 22
kdenlive version: 21.12.3
Audio is available in editing mode and can be played, however, after the project is rendered/exported into a file, video is fine but there is no audio.
The attached screenshot shows the rendering configuration:

How can I resolve the issue?


text:
Restoring High Frame Rate Video After Forced Slow Motion Conversion
I'm dealing with a tricky situation involving high frame rate video recording and need some guidance. For an experiment, I needed to record a video at a high frame rate. The only way to achieve this on my iPhone was by using the slow motion setting, which records at 240 fps. However, the problem arose after transferring the video via AirDrop. The video automatically converted to 30 fps, and the slow motion effect became a permanent feature, eliminating the high frame rate I originally intended to capture. Additionally I have already lost the original video file, so I’m left only with the 30 fps version.
To salvage the situation, I used Flowframes to interpolate the video back to 240 fps. This step successfully restored the frame rate, but the video still retains the unwanted slow motion effect. My questions are:

How can I manually adjust the speed of the video to negate the slow motion effect and return it to normal playback speed? Is there a specific factor or method that should be used for this adjustment?

Could the conversion from 30 fps to 240 fps have impacted the video in a way that complicates restoring it to its original speed?

Any advice or solutions to revert my video to its intended high frame rate without the slow motion effect would be greatly appreciated!


text:
ffmpeg multiple png fade in/out, audio and overlay with filter_complex_script
I have a set of transparent png images, each image having a coresponding wav audio file (the audio files are of different lengths): 1.png, 1.wav, 2.png, 2.wav, etc...
There is also a 10 seconds looping background video, back.mp4.
The video should be as such:

an image fades-in for 2 seconds
the image is displayed while playing the coresponding audio
the image fades-out for 2 seconds

... and so on for all the images/audio files.
Finally, all of this should be overlaid on top of back.mp4, which needs to loop until the end.
I need to achieve this with a filter_complex_script file (that is automatically generated), preferrably putting everything in the file (using movie/amovie instead of command line parameters to ffmpeg). It should be done also without having to calculate the lengths of the audio files.
I am struggling with this for days, and having problems with looping the image. Either the fade doesn't show, or the concatenation of the image with the audio gets endless.
Is it even possible to achieve?

using filter fade=out:st=.... it needs a start position of fading out, so you have to calculate length of audio. and so on for length of png.

Details needed: what's the output framerate? Are all audio expected to be at least 4+ seconds long? Are all images the same resolution, will they have to be scaled, altered somehow?

authors: Баяр Гончикжапов


text:
FFMPEG - scale and chromakey not working together?
video1.mp4 1280×720, with a green area
video2.mp4 1280×720
I need to reduce video2.mp4 to 640x480 and insert what fits into the green area of video1.mp4
I went through all the options that I found on the Internet, but they don't work in my case.
An example that reduces, but the chromakey does not work:
ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0]colorkey=0x00b200:0.02:0.03[v0];[1]scale=800:640[v1];[v0][v1]overlay[out]" -codec:v libx264 -crf 23 -preset slow -map "[out]" -map 0:a result.mp4
Please tell me how to fix it.

try to increase similarity 0.02 to 0.2 and, maybe, reverse [v1][v0]overlay?

"similarity 0.02 to 0.2", - it didn't help. If reverse [v1][v0]overlay then the chromakey is triggered, but the output video is cropped to size 640x480.

tested by this way ffmpeg -i "switch.mp4" -i "input 1.mp4" -filter_complex " [0]colorkey=#5fc502:0.1:0[v0]; [1]scale=640:480,pad=1280:720:-1:-1[v1]; [v1][v0]overlay " -c:v h264_nvenc -cq 23 -c:a copy -t 5 /tmp/output.mkv

authors: Баяр Гончикжапов


text:
ffmpeg: smooth zoompan with no jiggle
I'm trying to apply a smooth zoom effect to my 5-second video using FFmpeg, but I'm encountering a shaky/jittery zoom.
Here's the command I'm using:
ffmpeg -i island_zoom.mp4 -vf "zoompan=z='min(max(zoom,pzoom)+0.0015,1.5)':d=1:x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':s=1080x1920:fps=30" -preset ultrafast output_zoom.mp4

Input: island_zoom.mp4
Output: output_zoom.mp4
I am aware of the ffmpeg bug #4298. According to the comments, it's possible to upscale the resolution before applying the zoom, for example, using scale=8000x4000 before the zoompan filter. However, I'm trying to avoid this approach because I have a large number of videos to process, and adding this extra scaling step would significantly increase the processing time.
Can anyone suggest an effective workaround? Thanks for help!

pad to square 1920x1920, then zoom to divisible by 2 value:
ffmpeg -loop 1 -t 5 -r 30 -i img.jpg -vf "
format=yuv444p,
pad=1920:1920:-1:-1,
zoompan=z='trunc(iw*10*(1+0.0015*in)/2)*2/(iw*10)'
:d=1
:x='iw/2-iw/zoom/2'
:y='ih/2-ih/zoom/2'
:s=1920x1920
:fps=30,
crop=1080
" out.mp4

tested with jpg only


text:
Rotate Video Metadata - ffmpeg vs ExifTool
I created an ffmpeg script to rotate -90 some videos that I record on my Sony A7iii, with the camera vertically, this makes these videos display correctly.
the script is this:
ffmpeg -i input.mp4 -c:v copy -metadata:s:v:0 rotate=-90 output.mp4

The issue is that for some reason all the output files are having variations of 2 frames, which causes me problems when I use these files as Proxy in Premiere Pro.
For example, if the original file has a Media Duration (in Premiere Pro) of 00:00:32:12, the output file for some reason will have a Media Duration of 00:00:32:10.
Another very curious point is that the file generated by ffmpeg always has a size variation, whereas the file generated by ExifTool has no variation at all...
ExifTool
Using this ExifTool command I can do the same conversion and I don't have this frame shifting problem:
exiftool -m -P -overwrite_original -api LargeFileSupport=1 -rotation=90 movie.mp4

But I would still like to be able to get the proper result using ffmpeg, because doing the work through ffmpeg would make my workflow easier.
Any idea why the problem is occurring?
Edit
I still haven't managed to get it to work through FFMPEG, does anyone have any ideas?


text:
Rotate video without re-encoding - problems
I created an FFMpeg script to rotate -90 some videos that I record on my Sony A7iii, with the camera vertically, this makes these videos display correctly.
the script is this:
ffmpeg -i input.mp4 -c:v copy -metadata:s:v:0 rotate=-90 output.mp4

The issue is that for some reason all the output files are having variations of 2 frames, which causes me problems when I use these files as Proxy videos in Premiere Pro.
For example, if the original file has a Media Duration (in Premiere Pro) of 00:00:32:12, the output file for some reason will have a Media Duration of 00:00:32:10.
The files were written in XAVC 4K with 23,976 fps
How can I fix this problem?
I've already tried adding -r 23.976 to the code to force the frame rate, but that didn't help either.
The crucial point here is to rotate the video without re-encoding the file, for this reason, I took the approach of changing only the metadata, keeping the video stream untouched, re-encoding will cause a significant time penalty in my workflow.
EDIT
I did a test by completely copying the audio and video streams from my original file, the command line looked like this:
ffmpeg -i C2526.MP4 -c:v copy -c:a copy -metadata:s:v:0 rotate=-90 C2526-test.mov

I know that the .mov output would not return a valid file, since no conversion is being done, but it is necessary because ffmpeg does not accept copying the PCM stream from the camera file to the .mp4 container, so after generating it I rename it the extension to .mp4 for the file to be valid.
See that this command doesn't change anything other than the video rotation metadata, everything stays exactly the same, and even so the Media Duration has a variation of 2 frames...

Losslessly rotate QuickTime mov, mp4 or m4v 90° clockwise (270 CW is the same as -90 CCW. macOS Quick Look can also do the same):
exiftool -m -P -overwrite_original -api LargeFileSupport=1 -rotation=90 movie.mov

Wow! This works perfectly, I didn't know the ExifTool tool, thanks for the tip!
Do you know exactly how it works?
I would mainly like to know if it generates a new file or if it simply changes the metadata of the original file, because if a new file is generated, I believe it could be a negative point for the useful life of the storage devices.
I know that FFMPEG necessarily needs to generate a new file.

I was checking here through CrystalDiskInfo, it seems that the files really need to be rewritten... I believe that this is not very positive considering the useful life of the storage devices, but I think it is the only way...
Using ExifTool, the rotation works as expected, I'm not experiencing frame variation, however, for my flow it would still be very interesting to be able to resolve this issue with ffmpeg... if anyone knows a way to fix this "bug" or get around it, I'll still be happy to know.

exiftool -ver 10.89 and later edits the MatrixStructure tag of a .mp4, .m4v and .mov QuickTime movie. exiftool always re-writes the file.
exiftool -a -G1 -s -n -api LargeFileSupport=1 -MatrixStructure -Rotation movie.mp4
[Track2]        MatrixStructure                 : 1 0 0 0 1 0 0 0 1
[Composite]     Rotation                        : 0
[Track2]        MatrixStructure                 : 0 1 0 -1 0 0 1080 0 1
[Composite]     Rotation                        : 90
https://exiftool.org/forum/index.php?topic=6670.msg46745#msg46745

As I mentioned, the big problem with rewriting the file every time is the double wear on the storage units, this would reduce the useful life of a storage device by half. Isn't there a way to rotate by changing the metadata without rewriting the entire file?

exiftool by design rewrites the whole file when a change has been made to the metadata. Metadata may be restructured in such a way that it takes less space than in the original file, or so that some tags are stored at different offsets in the file. If you worry about wear, use a program that writes to XMP sidecar files. https://exiftool.org/faq.html#Q13

Thank you very much for the explanation and attention. Based on what you explained to me, I believe that XMP would not solve the problem, I need to open the rotated files within Premiere Pro, and I believe that with an external XMP this will not be possible.
Anyway, based on what you taught me and my needs, I created a script that avoids rewriting by directing the ExifTool output to another folder, so I can use the tool to unload my memory card with the videos already rotated.

I created a loop in powershell to execute this command: & $exiftoolPath -m -P -progress -api LargeFileSupport=1 -rotation=90 -o $outputDirectory $file.FullName -charset filename=cp1252

But I would still like to understand why ffmpeg generates this difference in frames, being able to use it would still be useful for me...


text:
Can ffmpeg encode only the beginning part of a video for cutting?
I am a really beginner in video editing. Sorry if this question is silly.
I heard videos consist of key frames (I-frames) and inter-frames (P-frames and B-frames).
Say I have a video of 10 seconds, and I ask ffmpeg to extract from 4.3 to 10.
ffmpeg -ss 4.3 -i input.mp4 output.mp4

From my observation, ffmpeg seems to reencode the whole output file.

Assume the original video has I frames at 1, 3, 5, 7, 9 seconds. I think ffmpeg can just insert a keyframe at 4.3, reencode frames from 4.3 to 5, and leave all data afterwards intact.
Is my idea realistic? Can ffmpeg do so?
I ask this is because if my input video is very long, my method can cut the video very fast.

heavy scripting
#!/bin/bash
cd /tmp
f="/mnt/public/upload/test/tg/2024-07-20_10꞉11꞉34.MOV"
# f="/mnt/public/upload/test/tg/2024-07-19_13꞉39꞉18.mp4"
t="tmp.mp4"
o="output.mp4"

IFS=, read vid wid hei sar fmt fps tbn <<< $(ffprobe -v 0 -select_streams v:0 -show_entries stream=codec_name,width,height,sample_aspect_ratio,r_frame_rate,time_base,pix_fmt -of csv=p=0 "$f")
if [ "$sar" = "N/A" ]; then sar=1; fi
tbn=${tbn#*/}
echo $vid $wid $hei $sar $fmt $fps $tbn

beg=4.3
end=$( echo "$beg + 10" | bc -l )
ofs=$(ffprobe -read_intervals ${beg}%${end} -skip_frame nokey -select_streams v:0 -show_entries frame=pts_time -of default=nw=1:nk=1 "$f" -v 16 | head -n 2 | tail -n 1)
echo "---> convert from $beg to $ofs"
ffmpeg -noautorotate -ss $beg -to $ofs -i "$f" -c:v $vid -video_track_timescale $tbn -an "$t" -y -hide_banner

echo "file '$t'
file '$f'
inpoint $ofs" > 1.txt
ffmpeg -f concat -safe 0 -i 1.txt -ss $beg -i "$f" -map 0:v:0 -map 1:a:0 -c copy "$o" -y -hide_banner
mpv --no-config --keep-open "$o"

this ↑ variant doesn't work sometimes with mp4, so you can try variant with using temporary mkv segments ↓
#!/bin/bash
cd /tmp
# f="/mnt/public/upload/test/tg/2024-07-20_10꞉11꞉34.MOV"
f="/mnt/public/upload/test/tg/2024-07-19_13꞉39꞉18.mp4"
t="tmp.mkv"
o="output.mp4"
beg=4.3
ffmpeg -i "$f" -c copy -an -f segment -segment_times $beg -reset_timestamps 1 tmp_%d.mkv -hide_banner -y
echo "---> convert tmp_0.mkv from $beg"
vid=$(ffprobe -v 0 -select_streams v:0 -show_entries stream=codec_name -of csv=p=0 "$f")
ffmpeg -noautorotate -ss $beg -i "tmp_0.mkv" -c:v $vid "$t" -y -hide_banner
echo "file '$t'
file 'tmp_1.mkv'
" > 1.txt
ffmpeg -f concat -i 1.txt -ss $beg -i "$f" -map 0:v:0 -map 1:a:0 -c copy "$o" -y -hide_banner
mpv --no-config --keep-open "$o"

Your scripting skill is awesome! Why doesn't ffmpeg implement this strategy by default? What is the drawback?


text:
ffmpeg filter_complex multiple overlays - why are they not syncing properly
I'm setting up live streaming using three overlaid inputs in ffmpeg. O:v is a webcam directly connected to the streaming box (a headless linux box) while 2:v and 3:v are ultra-low latency UDP streams from two remote R-Pis.
The filter_complex argument I'm using (that works, with all three video inputs perfectly in sync) is [0:v] fifo [v1];[2:v] fifo [v2]; [3:v] fifo [v3];[v1]setpts=PTS-STARTPTS[sync1];[v2]setpts=PTS-STARTPTS[sync2]; [v3] setpts=PTS-STARTPTS+5/TB [sync3] ;[sync1][sync2] overlay=x=W-w:y=H-h [out1];[out1][sync3] overlay=x=0:y=H-h [vfinal]
What I don't understand is why I am having to add +5/TB to [sync3] via setpts (in bold above)? If I omit that, [3:v] is synced in to the stream noticeably ahead of the other two inputs. My best guess is that this may be to compensate for the processing time of the first overlay [sync1][sync2] which is introducing extra latency into that part of the stream.
Importantly - the value of the extra pts compensation has to be dialed in by hand and will be specific to the underlying hardware used. So the question is, is there a way of getting ffmpeg to calculate this automatically so I don't have to worry about keeping it dialed in, in production?
Thanks for reading! I'm at the limit of the ffmpeg docs here...


text:
FPV-Camera Input in Black and White / losses Color on conversion with FFMPEG
so we're working on our end-of-school project and it's an FPV-Drone with an Analogue Camera on it. Plan is to send the video feed to a Raspberry Pi running an RTMP-Server from where a Phone-Application can view the live Video of the camera.
To convert this analogue Data from the camera we use a USB2.0 Grabber (this one).
To create the RTMP Stream from the converted USB-Input we use FFMPEG with the following command:

fmpeg -f v4l2 -input_format yuyv422 -i /dev/video0 -c:v libx264 -crf 20 -preset ultrafast -b:v 2000k -fflags nobuffer -rtmp_live live -f flv rtmp://192.168.8.107:554/live/stream

It works fine, but the main problems at the moment are:

the video is in Black and White
B&W Image Stream
the stream has a delay of 10-20s depending on the network

When I'm using the official Software provided by the Reseller I had the same problem, but as soon as it set PAL/BDGHI in the Settings the colour was shown correctly:
Settings and Color Image in official software
the video is in Black and White
Does anyone know what settings there are to correctly decode the feed from the Camera and send the video with the colour over RTMP? I don't know if this is the right place to ask this question, but I'm running out of ideas and every single decoder I have tried apart from the ones I'm currently using does not work.
Any help is greatly appreciated :)
Nevermind someone on Reddit found the problem:
https://www.reddit.com/r/ffmpeg/comments/18or1xt/fpv_camera_input_in_black_and_white_help/

Nevermind someone on Reddit found the problem: https://www.reddit.com/r/ffmpeg/comments/18or1xt/fpv_camera_input_in_black_and_white_help/


text:
Why does removing an audio track in Handbrake reduce volume?
I am transcoding a couple of blu-ray videos using Handbrake (v1.2.2). I play the resulting MKV containers mainly using Video LAN Client (VLC).
Often a blu-ray has several audiotracks that are basically identical in content, but different encoding. Since I'm transcoding all audio to AAC, all but one appear redundant to me, and I'd like to get rid of them. Doing so, however, severely reduces volume of the remaining tracks, in the order of magnitude of 10 to 12db! I'd like to understand this behaviour.
For instance there are four tracks in two languages, each language in DTS and DTS-HD MA. If I keep all tracks (transcoding them to AAC), the play-back volume is preserved. If in Handbrake (under the Audio/Track List section) I delete any one of the four tracks ("delete" meaning exempt from transcoding), the transcode result plays back with each of the remaining three audio tracks with reduced volume. Since this appears to be global, I thought maybe a global replay gain tag was reset. But checking the MKV container with MKVToolNix I cannot find any replay gain tag, neither global, nor track-specific, in the result of any of the experiments. If I do include all tracks, however, such that the full volume is retained, I can afterwards use MKVToolNix to remove (re-multiplex) the undesired audio without affecting volume - a slightly laborious workaround, but I really would like to understand what's going on.
The Handbrake settings for the AAC are the default (Bitrate 192kb/s, 5.1ch, sample rate as in source, extra gain and DRC off. Interestingly, I found the extra gain setting, when using it, does not effect the volume.
Edit:
I found the behaviour to be not as consistent as I thought, in fact, I've come to believe that I found a bug. Apparently, leaving the extra gain off (it is displayed as 0dB), does not leave the volume untouched, but in fact applies a more or less random gain (apparently triggered by deleting audio tracks), as can be confirmed in the activity log. Activating the extra gain, and explicitly setting it to 0dB makes Handbrake behave as expected.
Since StackExchange is not a place for bug reports, I wonder if I should delete this question? Please advise.


text:
FFMPEG: Remove entirely blue frames from a video
I'm trying to help my father who digitizes various video and audio tape formats.
It appears that occasionally digitization hard- and software does not receive enough information from a frame and the output result turns out to be a solid blue frame.
I'm a complete newbie in this type of thing but I found out FFMPEG might help me. I want to write a command that takes input .mp4 video, detects frames with a certain hex color and removes them from the output video (frames should be completely ignored/skipped/trimmed, audio must be synced as well).
I've tried using free ChatGPT (I know, I know..) but no luck so far.
I'm not a power user by any means.
Help please!


text:
Audio missing from clips, standard solution not working
Adobe Premier, Windows 10: I accidentally deleted some audio from some clips I had edited in the timeline by just moving one clip's audio over the other clip's audio in the same audio lane. I have looked for solutions, and the popular answer seems to be to use match frame and then hit "." This will reproduce the same clip with no audio on my timeline. Frustrating! I can find the original clip in the project, and it still has audio, and I could redo all the editing I have done so far, but that would take hours. Is there an actual solution to this issue? Am I pressing "F ." incorrectly? I've also tried "Generate audio waveform" - no luck. I refuse to believe that accidentally dragging one track's audio over another track's is such an irreversible mistake in such a high end program.
Please help


text:
How to insert alternate frames from multipule video sources to create a single output file using ffmpeg
I have a 4K 50fps video file that is split into 4 quadrants (only 3 are needed). It is generated by a slow motion camera running at 150 FPS. Each of the quadrants contains the same image shifted by a frame.
I want to cut out the 3 sections of the picture, take a frame from each image and insert them sequentially. This will result in a 1920x1080 50FPS clip 3 times a long as the original, but the motion in the picture will run slower.
I know how to cut the sections from the image, but I'm struggling to find a filter to insert the frames sequentially.
ffmpeg -i TriMotion.ts -filter:v "crop=1920:1080:0:0" out.mp4
ffmpeg -i TriMotion.ts -filter:v "crop=1920:1080:1920:0" out.mp4
ffmpeg -i TriMotion.ts -filter:v "crop=1920:1080:0:1080" out.mp4
I have tried the following, but the output is normal speed;
ffmpeg -i TriMotion.ts -filter_complex "[0]crop=1920:1080:0:0[v1];[0]crop=1920:1080:1920:0[v2];[0]crop=1920:1080:0:1080[v3];[v1][v2][v3] interleave=nb_inputs=3"  -c:v libx265 out.mov

For anyone else looking for this, I think that I have worked it out.
You need to change the frame numbers of the incoming sources before they are combined:
ffmpeg -i TriMotion.ts -framerate 50 -filter_complex "[0]crop=1920:1080:0:0,setpts=(PTS*3)-2[v1];[0]crop=1920:1080:1920:0,setpts=(PTS*3)-1[v2];[0]crop=1920:1080:0:1080,setpts=(PTS*3)[v3];[v1][v2][v3] interleave=nb_inputs=3"  -c:v libx265 out.mov
This command sets the frame numbers in increments of 3 before the interleave filter combines them.


text:
Ffmpeg input seeking - "Invalid NAL unit size"
I'm trying to use ffmpeg to get a 10-second clip from the middle of a video. The execution time of the command is important, that's why I've decided to use combined input & output seeking (as illustrated here). The input video file is a CMAF with fragmented MP4, duration of 10 minutes.
I'm testing on a Mac, Ffmpeg version is 6.1.1.
This is the command that I'm using:
ffmpeg -nostdin -y -ss 290 -copyts -start_at_zero -i https://devcdn.flowplayer.com/5f07362e-c358-41d0-857a-c64302a3fcc9/cmaf/17bdb16d-71d1-414c-a291-a028bd45b9ec/playlist_360.m3u8 -ss 300.0 -t 10 -vcodec libwebp -lossless 0 -quality 60 -compression_level 2 -loop 0 -an -sn output.webp

Result: no output file is created.
From what I understand it fails to seek position "290" in the video, probably due to "Invalid NAL unit size" errors.
Here's the output:
ffmpeg version N-106797-g580fb6a8c9-tessus Copyright (c) 2000-2022 the FFmpeg developersbuilt with Apple clang version 11.0.0 (clang-1100.0.33.17)configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplaylibavutil      57. 24.101 / 57. 24.101libavcodec     59. 27.100 / 59. 27.100libavformat    59. 23.100 / 59. 23.100libavdevice    59.  6.100 / 59.  6.100libavfilter     8. 37.100 /  8. 37.100libswscale      6.  6.100 /  6.  6.100libswresample   4.  6.100 /  4.  6.100libpostproc    56.  5.100 / 56.  5.100Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://devcdn.flowplayer.com/5f07362e-c358-41d0-857a-c64302a3fcc9/cmaf/17bdb16d-71d1-414c-a291-a028bd45b9ec/playlist_360.cmfv':Metadata:major_brand     : isomminor_version   : 1compatible_brands: isomavc1dashcmfccreation_time   : 2024-01-30T07:41:03.000000ZDuration: 00:09:56.54, start: 0.083333, bitrate: 458 kb/sStream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x360 [SAR 1:1 DAR 16:9], 1 kb/s, 24 fps, 24 tbr, 90k tbn (default)Metadata:creation_time   : 2024-01-30T07:41:03.000000Zhandler_name    : ETI ISO Video Media Handlervendor_id       : [0][0][0]ffmpeg version 6.1.1-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[hls @ 0x7fc7bb904280] Skip ('#EXT-X-VERSION:6')
[hls @ 0x7fc7bb904280] Opening 'https://devcdn.flowplayer.com/5f07362e-c358-41d0-857a-c64302a3fcc9/cmaf/17bdb16d-71d1-414c-a291-a028bd45b9ec/playlist_360.cmfv' for reading
    Last message repeated 2 times
Input #0, hls, from '[**]/playlist_360.m3u8':
  Duration: 00:09:56.46, start: 0.083333, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 640x360 [SAR 1:1 DAR 16:9], 1 kb/s, 24 fps, 24 tbr, 90k tbn (default)
    Metadata:
      variant_bitrate : 0
      compatible_brands: isomavc1dashcmfc
      handler_name    : ETI ISO Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Elemental H.264
      major_brand     : isom
      minor_version   : 1
      creation_time   : 2024-01-30T07:41:03.000000Z
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> webp (libwebp))
[hls @ 0x7fc7bb904280] Opening 'https://devcdn.flowplayer.com/5f07362e-c358-41d0-857a-c64302a3fcc9/cmaf/17bdb16d-71d1-414c-a291-a028bd45b9ec/playlist_360.cmfv' for reading
    Last message repeated 2 times
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1772342253 > 1534).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1538
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1977545460 > 1481).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1485
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1694403391 > 1582).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1586
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1404850266 > 1661).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1665
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (703351242 > 1680).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1684
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-836978648 > 1751).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1755
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (752797651 > 1867).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1871
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1831058223 > 1833).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 1837
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1238958831 > 2067).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2071
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (435683248 > 2090).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2094
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (2136335178 > 2229).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2233
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1468707300 > 2203).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2207
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (482758774 > 2402).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2406
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1079612217 > 2417).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2421
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-608087491 > 2546).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2550
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-1457748625 > 2527).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2531
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1933919710 > 2734).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2738
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1004643870 > 2803).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2807
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-207765435 > 2988).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2992
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-196888537 > 2306).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2310
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1118966683 > 2620).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2624
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1325583054 > 2715).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2719
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-2003602869 > 2906).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2910
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1666330272 > 3085).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 3089
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (-742329993 > 2593).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2597
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (1326266794 > 2347).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2351
[NULL @ 0x7fc7bb804f40] Invalid NAL unit size (2459776 > 2155).
[NULL @ 0x7fc7bb804f40] missing picture in access unit with size 2159
[https @ 0x7fc7ba022a00] Opening 'https://devcdn.flowplayer.com/5f07362e-c358-41d0-857a-c64302a3fcc9/cmaf/17bdb16d-71d1-414c-a291-a028bd45b9ec/playlist_360.cmfv' for reading
[...]
[vost#0:0/libwebp @ 0x7fc7bbb05780] No filtered frames for output stream, trying to initialize anyway.
Output #0, webp, to 'output.webp':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0(und): Video: webp, yuv420p(progressive), 640x360 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 24 fps, 1k tbn (default)
    Metadata:
      variant_bitrate : 0
      compatible_brands: isomavc1dashcmfc
      handler_name    : ETI ISO Video Media Handler
      vendor_id       : [0][0][0][0]
      creation_time   : 2024-01-30T07:41:03.000000Z
      major_brand     : isom
      minor_version   : 1
      encoder         : Lavc60.31.102 libwebp
[out#0/webp @ 0x7fc7bbb04900] video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[out#0/webp @ 0x7fc7bbb04900] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A

What I've tried so far:

Downloaded the input file to a local directory and used it as input to ffmpeg - same results.

Used the mp4 file from the playlist directly as an input to ffmpeg - worked but the execution time is very slow

Emmited the input seeking part (-ss 290 -copyts -start_at_zero) from the command - worked but also very slow in terms of execution time

Do you have any ideas on why I'm getting "Invalid NAL unit size" and how to make the command work with input seeking?

FFMpeg indeed creates an empty file with said error if I provide URL. However once I download the file, e.g. using youtube-dl which creates playlist_360.mp4, FFMpeg has no issues to encode into webp output file. I have a following FFMpeg version: ffmpeg version N-113482-g36a986d9a1 Copyright (c) 2000-2024, including enabled libwebp.


text:
How do I add a "border" in showwaves filter with FFMPEG?
Im attempting to make a wave video script in bash that allows the user to input an audio file and a background and displaying a wave filter over that image, and 1 that has a border around it so it doesnt look look so ugly being overlayed on the video. My solution to getting this border is adding another show waves filter first that is bigger than the second show waves filter. Getting 1 wave filter was relatively easy and I was able to do that without much a problem.
function designwavevideo() {
        outputpath="/storage/emulated/0/Pictures/Wave Audio Videos/"
        read -p "Enter Audio File: " audiofile
        read -p "Enter Background File: " backgroundfile
        backgroundwidth=$(ffprobe -v error -select_streams v -show_entries stream=width -of csv=p=0:s=x "$backgroundfile")
        backgroundheight=$(ffprobe -v error -select_streams v -show_entries stream=height -of csv=p=0:s=x "$backgroundfile")
        # read -p "Enter Border Wave Thickness:" borderwavethickness
        read -p "Output File Name:" outputfilename
        let widthwavethickness=$backgroundwidth+$borderwavethickness
        let heightwavethickness=$backgroundheight+$borderwavethickness
        ffmpeg -y -i "${audiofile}" -i "${backgroundfile}" -threads 16  -filter_complex "[0:a]aformat=channel_layouts=mono,showwaves=size=${backgroundwidth}x${backgroundheight}:mode=cline:rate=25:draw=full:colors=#43fa62[v];[1:v][v]overlay=format=auto:x=(W-w)/2:y=(H-h)/2,format=yuv420p[outv]" -map "[outv]" -c:a aac -map 0:a -c:v libx264 -crf 28 -preset veryslow -b:v 0 -shortest "${outputpath}${outputfilename}"
}

But Adding 2 waves to the same video is proving to be a bit complicated for me. As shown by the snippet below. I tried to add another wave filter and set the overlays but that dint work so now I hit a stump and im not quite sure how to go about fixing this.
function designwavevideo() {
        outputpath="/storage/emulated/0/Pictures/Wave Audio Videos/"
        read -p "Enter Audio File: " audiofile
        read -p "Enter Background File: " backgroundfile
        backgroundwidth=$(ffprobe -v error -select_streams v -show_entries stream=width -of csv=p=0:s=x "$backgroundfile")
        backgroundheight=$(ffprobe -v error -select_streams v -show_entries stream=height -of csv=p=0:s=x "$backgroundfile")
        read -p "Enter Border Wave Thickness:" borderwavethickness
        read -p "Output File Name:" outputfilename
        let widthwavethickness=$backgroundwidth+$borderwavethickness
        let heightwavethickness=$backgroundheight+$borderwavethickness
        ffmpeg -y -i "${audiofile}" -i "${backgroundfile}" -threads 16 -filter_complex "[0:a]aformat=channel_layouts=mono,showwaves=size=${widthwavethickness}x${heightwavethickness}:mode=cline:rate=25:draw=full:colors=#000000[v1];[0:a]aformat=channel_layouts=mono,showwaves=size=${backgroundwidth}x${backgroundheight}:mode=cline:rate=25:draw=full:colors=#43fa62[v2];[1:v][0][v1]overlay=format=auto[o1];[o1][v2]overlay=format=auto[o1]" -map "[outv]" -c:a aac -map 0:a -c:v libx264 -crf 28 -preset veryslow -b:v 0 -shortest "${outputpath}${outputfilename}"
}


text:
How to fix an mp4 when 1/4 of the video is in one orientation and the other 3/4 is rotated 90 degrees?
I have an mp4 video shot on an iPhone. Part way through the video the user rotated the camera 90 degrees and kept shooting. The first 1/4 of the video is right side up, there is a small section where the camera rotates, and then the last 3/4 of the video is rotated by 90 degrees. All one video clip.
Is there a way to fix the video without using an iPhone? I don't have an iPhone, just an Ubuntu 22.04 computer. ffmpeg? VLC media player? Openshot? Do I have to use an iPhone?
There are lots of tutorials for rotating an entire video using these tools, but I can't find anything that shows how to rotate the video starting at a certain time stamp.
I could add the video to Openshot, cut the clip into three pieces, the good stuff, the frames showing the rotation, and the rotated stuff. Then rotate the rotated clip correctly, and then join the first and third clips together and throw out the frames showing the rotation in progress. There would be a rough transition from one clip to the other.
Is there any open source program that can read each frame's metadata and correct the rotation angle on each frame? Can ffmpeg do that?

pad, rotate animation
ffmpeg \
-ss 0.0 -to 2.5 -i input.mp4 \
-ss 2.5 -to 3.0 -i input.mp4 \
-ss 3.0 -to 5.5 -i input.mp4 \
-filter_complex "
[0:v]pad='max(iw,ih)':'max(iw,ih)':-1:-1[a];
[1:v]pad='max(iw,ih)':'max(iw,ih)':-1:-1,rotate='lerp(0,PI/2,t/0.5)'[b];
[2:v]pad='max(iw,ih)':'max(iw,ih)':-1:-1,rotate=PI/2[c];
[a][0:a][b][1:a][c][2:a]concat=n=3:v=1:a=1
" output.mp4

t/0.5 → where 0.5 is duration of second part of input

Thanks for your help. Can you give me a little more explanation of what your command does?

Split input to 3 parts: normal, rotation from 0° to 90°, rotated. Concat them.

or you can try this way: ffmpeg -i input.mp4 -filter_complex " [0:v]pad='max(iw,ih)':'max(iw,ih)':-1:-1, rotate='if(lt(t,2.0),0,if(between(t,2.0,2.5),lerp(0,PI/2,(t-2.0)/0.5),PI/2))' " -c:a copy output.mp4

authors: Баяр Гончикжапов