4

Why does ffmpeg produce variable frame rate output?

I have an input file, that when I run it through MediaInfo it tells me it's a constant frame rate video:

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@High
HDR format                               : Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible / SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 h 39 min
Bit rate                                 : 16.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.083
Stream size                              : 11.5 GiB (90%)
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : BT.2020
Mastering display luminance              : min: 0.0000 cd/m2, max: 1000 cd/m2

I try to re-encode the video using ffmpeg using this command line:

ffmpeg -i input.mkv -map 0 -c:v libx265 -crf 24 -preset slow output.mkv

After ffmpeg has had its hands on the video it's now a variable frame rate output according to MediaInfo (for the examples I canceled the encoding shortly after starting it, but even if I let it run through it still reports variable frame rate):

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 2 s 172 ms
Bit rate                                 : 550 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.941 FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 0)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.003
Stream size                              : 146 KiB (33%)
Writing library                          : x265 3.5+37-07b011400:[Windows][GCC 11.2.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=24.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=0 / chromaloc-bottom=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant

I read on this page that if I specify an fps filter I should be able to force constant frame rate:

https://trac.ffmpeg.org/wiki/ChangingFrameRate

So I try that with the following command line:

ffmpeg -i input.mkv -vf fps=source_fps -map 0 -c:v libx265 -crf 24 -preset slow output.mkv

This still produces an variable frame rate output:

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 s 547 ms
Bit rate                                 : 386 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.917 FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 0)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.002
Stream size                              : 72.9 KiB (25%)
Writing library                          : x265 3.5+37-07b011400:[Windows][GCC 11.2.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=24.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=0 / chromaloc-bottom=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant

And a third thing I've tried is using the vsync parameter, which according to https://ffmpeg.org/ffmpeg.html#Advanced-options should be able to enforce constant frame rate.

This commandline: ffmpeg -i input.mkv -vsync cfr -map 0 -c:v libx265 -crf 24 -preset slow output.mkv

also produces a variable frame rate output:

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 s 838 ms
Bit rate                                 : 469 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.939 FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 0)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.002
Stream size                              : 105 KiB (29%)
Writing library                          : x265 3.5+37-07b011400:[Windows][GCC 11.2.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=24.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=0 / chromaloc-bottom=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant

How can I re-encode my video using ffmpeg and retain CFR?

dropbear
  • 143
  • 1
  • 4
  • 2
    since adaptive B-frame placement is among *the* killer features of x265, I'm not sure which problem an adaptive frame rate really poses, as any decoder will have to be able to deal with that kind of adaptivity, anyways – this is a bit of speculation, but the encoder probably just sees multiple identical frames in succession and merges them to one with a multiple of the individual frame's duration? So, what's the problem you're solving here by trying to enforce constant frame rate? – Marcus Müller May 01 '22 at 11:27
  • @MarcusMüller Thank you for trying to help make sense of what probably happens. I would however like to have the same frame rate as the original material. If your speculation is correct, I understand it will increase the final size without providing any additional quality, but I would still like to have it if it's possible. And I know it's possible since if I try to encode the movie in Handbrake I'm able to select CFR and get an output file with CFR. – dropbear May 01 '22 at 14:20
  • I can't reproduce the problem. Can you reproduce it using synthetic input video like: `ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=(24000/1001):duration=5 -c:v libx265 -crf 24 -pix_fmt yuv420p input.mkv`? – Rotem May 01 '22 at 15:15
  • @Rotem Running your command line produces a CFR output for me. – dropbear May 01 '22 at 16:02
  • I want to help you but I can't reproduce the problem. Can you point me to a video file that reproduces the problem, so I can my suggested solution (instead of guessing). Try forcing the timestamps using `setpts` filter: `ffmpeg -i input.mkv -c:v libx265 -vf "setpts=N*1001/24000/TB" -crf 24 output.mkv` – Rotem May 01 '22 at 18:04
  • @Rotem Your `setpts` filter unfortunately also produces a VFR output. The video I'm using is a Bluray movie I ripped and am re-encoding, so it's not something I can share. – dropbear May 02 '22 at 15:48
  • @MarcusMüller Is b-adapt the reason a CFR gets forced to VFR? – Enigma Jan 24 '23 at 14:20
  • @dropbear Should `fps=source_fps` actually be `fps=fps=source_fps` ? (http://ffmpeg.org/ffmpeg-filters.html#toc-Examples-80) – Enigma Jan 24 '23 at 14:21
  • 1
    @Rotem If you have some media which has black frames at the beginning or maybe the end, you may be able to reproduce this. It seems the black frames can be the cause of the VFR tag, even if the media is mostly CFR. – Enigma Jan 24 '23 at 19:48
  • @Enigma can you share a sample video file that reproduces the issue? – Rotem Jan 25 '23 at 18:16
  • See [Changing the frame rate](https://trac.ffmpeg.org/wiki/ChangingFrameRate) that says this depends on the output format. MP4 for example defaults to constant frame rate. The parameter to use is `-r`, but again, its effect depends on the output format. – harrymc Jan 26 '23 at 09:01
  • @harrymc Yes I have seen that. In this case always MKV. I have all three of these on one encode and it is still VFR (From the first 4 seconds of black frames at start of video). `-vsync cfr` , `-x265-params fps=24000/1001:[other_params]` , `-filter:v fps=fps=source_fps`. If I split the encode by chapter for example (mkvtoolnix), I get 8 samples, 7/8 of which ARE CFR. It is only the first sample which is VFR. If I run blackdetect I see from 0 to 4.009 seconds. If I cut that bit out, only that bit is VFR. So simply by trimming I can make the whole encode CFR but I should not have to do this. – Enigma Jan 26 '23 at 11:28
  • I'd guess the input video has a variable frame rate, the codec supports variable frame rate encoding—assuming you haven't specified a variable frame rate through any command line options—or the output frame rate is not an exact multiple of the input frame rate. Can you confirm those all three to not be true and then see if your problem persists? – Vomit IT - Chunky Mess Style Jan 26 '23 at 23:56
  • The encoding in the first 4 seconds of this video seems to differ from the rest. – harrymc Jan 28 '23 at 16:29
  • Try [MKVToolNix](https://www.videohelp.com/software/MKVToolNix). See also the double-copy trick ([link](https://www.reddit.com/r/ffmpeg/comments/j9cig6/ffmpeg_transforms_my_constant_framerate_to_a/)) and [this post](https://superuser.com/questions/1611907/can-i-use-ffmpeg-to-convert-a-variable-frame-rate-video-to-constant-frame-rate-w). – harrymc Jan 29 '23 at 09:51
  • Solved it on my end effectively. Standalone tests with most of the above parameters worked to retain CFR but there is possibility of subsequent ffmpeg actions (not to video stream) which somehow affects the framerate of the video stream. Actions such as re-ordering or pulling streams out. – Enigma Jan 29 '23 at 12:25
  • @Rotem I cannot, as it was through attempting to provide such a clip that I discovered the source of the issue. Thank you though. – Enigma Jan 29 '23 at 13:41

2 Answers2

2

Solved the same issue on my end. Perhaps the circumstances are the same as the OP. Using any or all of these parameters should work: -vsync cfr, -x265-params fps=24000/1001:[other_params], -filter:v fps=fps=source_fps.

The issue I saw was that a subsequent ffmpeg processing command (not on the video stream) ended up affecting the video stream framerate data anyway. If anyone has any idea what would do this or why, please feel free to share. Seems like a bug to me.

Enigma
  • 3,371
  • 11
  • 48
  • 76
  • I've now tried the suggested arguments, both all of them individually (and the fps filter like previously, and like suggested with `fps=fps=24000/1001`), and a couple of them in conjunction. All of my attempts produces a VFR result. Also, in my attempts now I don't use any other arguments, simply the ones I mentioned before. I tried removing everything else and just keeping e.g. `-c:v libx265 -x265-params fps=24000/1001` as well, with the same VFR result. So pretty sure nothing is overriding the provided arguments. – dropbear Jan 30 '23 at 16:42
  • @dropbear After running the ffmpeg command, remux it with MKVToolNix and mediainfo should update accordingly. ffmpeg on it's own does not update these values accurately. – Enigma Jan 30 '23 at 16:53
  • That apparently does it. So maybe my issue was a combination of specifying the arguments incorrectly on my initial encode, and then just canceling the encode on subsequent attempts and assuming that it should be correctly identified as CFR. – dropbear Jan 30 '23 at 19:46
  • @dropbear Yeah cancelling is not a good idea to get accurate data. If anything you'd want to use `-t 00:00:30` or something similar for the first 30 seconds. – Enigma Feb 01 '23 at 16:38
1

The initial issue here is (most probably) that you trust the exif data of your original video file. MediaInfo does NOT analyze a video in any way, but it will show the exif data of your file as it was written by the original software used.

ffmpeg is more "correct" in specifying that the video actually is VFR: although most of it is CFR (as I can read from your comments, which should maybe make their way into the question), there must be some part VFR, which ffmpeg will detect and specify in the output file.

That said, you'll need to do an explicit pull-up/pull-down in order to achieve CFR. There's a wiki page about changing frame rates. As the mkv format will by default produce VFR if nothing specific is specified, you'll have to explicitly state the framerate you wish to produce:

  • the fps filter should always produce CFR (see the same wiki page) if used correctly. According to the manual, in your case you should use -filter:v fps=fps=24000/1001, but you might have to tinker with the start_time and round values to get a result which is the same as your original (as detailed in the manual).
  • If this does not work for some possible weird bug, what I'd suggest is to produce first an mp4, which will default to CFR - if necessary using the -r output option. And then, copy that back to mkv using ffmpeg -i input.mp4 -c:v copy -c:a copy output.mkv, which is very fast as no re-encoding is needed.
1NN
  • 5,232
  • 1
  • 17
  • 37
  • So the meta data argument would explain why `fps=source_fps` would fail. But then specifying `fps=fps=24000/1001` like you're suggesting, should work, and it does not. These arguments exactly produces a VFR: `ffmpeg -i input.mkv -c:v libx265 -filter:v fps=fps=24000/1001 output.mkv` . I interpret your answer as the `round` and `start_time` should only be required to get the same fps as the original video, not required to produce a CFR. Admittedly, like I mentioned before, I am canceling the encode a few seconds in when testing now. Could that screw up the result? – dropbear Jan 30 '23 at 16:49
  • @dropbear 1) yes, cancelling the encode could make a difference. Can't you just truncate the original video before encoding? or use `-t 10` as output option to only encode the first 10 seconds? 2) try to output an mp4, see what happens – 1NN Jan 30 '23 at 20:44
  • Awarded bounty here since I can't award myself – Enigma Feb 01 '23 at 16:40
  • @Enigma thanks. Hope it can be helpful for someone in some way. – 1NN Feb 04 '23 at 08:19