VSE Interlaced Video FPS not being honored #51153

New Issue

Kory Prince · 2017-04-08T00:35:36+02:00

Kory Prince commented

2017-04-08 00:35:36 +02:00

System Information
Arch x64 Linux 4.9.8-1-ARCH

Blender Version
Working: 2.78a Hash: e8299c8
Broken: 2.78b Hash: 3c043732d3
Broken: 2.78c Hash: e92f235283

Short description of error

Blender Frame Rate is set to 29.97 fps.

Adding a Movie file of 1080i to Blender 2.78a and previous would add a movie and audio strip of the same (correct) correct length.

Adding the same file to Blender 2.78b and higher has the audio strip the correct length and the movie strip twice as long.

Here is the video stream information from ffprobe:

Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc

If I change Blender's Frame Rate to 59.94, then the strip is the correct length. It's like blender is now reading the tbr/tbc value instead of the fps of the video.

I certainly don't claim to be an expert on video encodings, and I am probably saying this wrong. But the behavior is definitely different.

I've attached a small clip to experiment with.

Edit: The original upload did not exhibit the problem. Please use the file bad_clip_2.mts in my comment below.

**System Information** Arch x64 Linux 4.9.8-1-ARCH **Blender Version** Working: 2.78a Hash: e8299c8 Broken: 2.78b Hash: 3c043732d3f Broken: 2.78c Hash: e92f2352830 **Short description of error** Blender Frame Rate is set to 29.97 fps. Adding a Movie file of 1080i to Blender 2.78a and previous would add a movie and audio strip of the same (correct) correct length. Adding the same file to Blender 2.78b and higher has the audio strip the correct length and the movie strip twice as long. Here is the video stream information from ffprobe: Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc If I change Blender's Frame Rate to 59.94, then the strip is the correct length. It's like blender is now reading the tbr/tbc value instead of the fps of the video. I certainly don't claim to be an expert on video encodings, and I am probably saying this wrong. But the behavior is definitely different. I've attached a small clip to experiment with. **Edit:** The original upload did not exhibit the problem. Please use the file bad_clip_2.mts in my comment below.

Kory Prince commented

2017-04-08 00:35:36 +02:00

Changed status to: 'Open'

Kory Prince commented

2017-04-08 00:35:36 +02:00

Added subscriber: @korylprince

Olly Funkster commented

2017-04-10 15:05:41 +02:00

Added subscriber: @Funkster-3

Olly Funkster commented

2017-04-10 15:05:41 +02:00

I can't replicate I'm afraid... I get the same result in 2.78a (same hash as you mention) as 2.78c - the clip is seen as 59.94 fps and if it's the first clip imported, the scene framerate is set to match. It could be that all of them are wrong, but they are at least consistent.

What version of ffmpeg do you have, and are your blender versions downloaded from blender.org / installed from repos / built from source?

I can't replicate I'm afraid... I get the same result in 2.78a (same hash as you mention) as 2.78c - the clip is seen as 59.94 fps and if it's the first clip imported, the scene framerate is set to match. It could be that all of them are wrong, but they are at least consistent. What version of ffmpeg do you have, and are your blender versions downloaded from blender.org / installed from repos / built from source?

Kory Prince commented

2017-04-10 16:03:08 +02:00

So... Yeah, apparently the clip I uploaded does not exhibit the same properties as the original file. I short, the original file works as I described. The clip, which I created with ffmpeg -i 00008.MTS -c copy -t 10 clip.MTS, has the problem in all of the versions I've tried.

ffmpeg version:

ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 6.3.1 (GCC) 20170109

All of the blender builds are directly from the blender website.

I have taken a second clip, produced with dd if=00008.MTS of=bad_clip_2.mts bs=10M count=1 that does exhibit the properties I reported. Please try it in versions 2.78a and 2.78b to see what I mean.

Adding the clip to 2.78a will add the clip correctly with the fps set to 29.97. Adding the clip to a fresh 2.78b will set the fps to 59.94 and add the clip "correctly". If another clip exists and the fps is set to 29.97, the the clip is added incorrectly with the video twice as long as the audio.

Sorry for the confusion. I didn't generate the clip until I was done with all my testing.

bad_clip_2.mts

So... Yeah, apparently the clip I uploaded does not exhibit the same properties as the original file. I short, the original file works as I described. The clip, which I created with `ffmpeg -i 00008.MTS -c copy -t 10 clip.MTS`, has the problem in all of the versions I've tried. ffmpeg version: ``` ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 6.3.1 (GCC) 20170109 ``` All of the blender builds are directly from the blender website. I have taken a second clip, produced with `dd if=00008.MTS of=bad_clip_2.mts bs=10M count=1` that **does** exhibit the properties I reported. Please try it in versions 2.78a and 2.78b to see what I mean. Adding the clip to 2.78a will add the clip correctly with the fps set to 29.97. Adding the clip to a fresh 2.78b will set the fps to 59.94 and add the clip "correctly". If another clip exists and the fps is set to 29.97, the the clip is added incorrectly with the video twice as long as the audio. Sorry for the confusion. I didn't generate the clip until I was done with all my testing. [bad_clip_2.mts](https://archive.blender.org/developer/F541517/bad_clip_2.mts)

Stephen Swaney commented

2017-04-10 20:30:26 +02:00

Added subscriber: @StephenSwaney

Stephen Swaney commented

2017-04-10 20:30:26 +02:00

second clip, produced with dd if=00008.MTS of=bad_clip_2.mts bs=10M count=1

I have not dived into the problem yet, BUT dd seems like a dubious tool for processing movie clips. Using dd on disk blocks is fine but there are no guarantees with formatted data. Corrections welcome.

> *second clip, produced with dd if=00008.MTS of=bad_clip_2.mts bs=10M count=1* I have not dived into the problem yet, BUT dd seems like a dubious tool for processing movie clips. Using dd on disk blocks is fine but there are no guarantees with formatted data. Corrections welcome.

Kory Prince commented

2017-04-10 21:35:50 +02:00

@StephenSwaney Oh, I agree completely. It's just the easiest way to get a small clip that exhibits the problem. It acts exactly like the larger (3.3GB), unmodified clip. I was using ffmpeg to create the clip, but whatever it does to the newly created file, the behavior is not the same.

My guess is MTS is similar to MP3s where there are "headers" every few bytes in the file so that it can be used for streaming/seeking, so while the very end of the clip might be garbage, the first part should be fine (blender plays the clip fine.)

I'm not trying to be like: https://xkcd.com/1172/ , I promise.

My real question is, what's the expected behavior? Should a 29.97 fps 1080i clip be interpreted in blender as 59.94 instead of 29.97? It wasn't before 2.78b, but now it is.

@StephenSwaney Oh, I agree completely. It's just the easiest way to get a small clip that exhibits the problem. It acts exactly like the larger (3.3GB), unmodified clip. I was using ffmpeg to create the clip, but whatever it does to the newly created file, the behavior is not the same. My guess is MTS is similar to MP3s where there are "headers" every few bytes in the file so that it can be used for streaming/seeking, so while the very end of the clip might be garbage, the first part should be fine (blender plays the clip fine.) I'm not trying to be like: https://xkcd.com/1172/ , I promise. My real question is, what's the expected behavior? Should a 29.97 fps 1080i clip be interpreted in blender as 59.94 instead of 29.97? It wasn't before 2.78b, but now it is.

Olly Funkster commented

2017-04-11 11:46:22 +02:00

I am full of questions.

Firstly, I also do not know what the desired behaviour should be - Blender does not have any way of representing interlaced frames internally (right? Bueller?) so it's got to do something destructive to the data. I would expect it to deinterlace by merging fields and end up at the target frame rate, rather than using each field as a frame, but that's just my preference.

Also the fact that the two test clips are handled differently kinda points to the original clip being nonstandard in some way... where did it come from? Straight from a video camera?

Anyway, what's odd is that I added bad_clip_2 while running in the debugger to see what's what, and ended up with a 333 frame strip at 29.97fps. This is a build that's based on master well after 2.78b (hash is f0cf15b), yet it exhibits the earlier behaviour.

I suspect it's a difference in some library or other rather than in the Blender source code itself, as the numbers appear during calls to avformat_open_input and avformat_find_stream_info (anim_movie.c, function startffmpeg).

Am I right in thinking that the blender builds on blender.org bundle libavformat etc. statically linked? ldd doesn't show any links to it, whereas it certainly does on the one I've built from source.

I am full of questions. Firstly, I also do not know what the desired behaviour should be - Blender does not have any way of representing interlaced frames internally (right? Bueller?) so it's got to do something destructive to the data. I would expect it to deinterlace by merging fields and end up at the target frame rate, rather than using each field as a frame, but that's just my preference. Also the fact that the two test clips are handled differently kinda points to the original clip being nonstandard in some way... where did it come from? Straight from a video camera? Anyway, what's odd is that I added bad_clip_2 while running in the debugger to see what's what, and ended up with a 333 frame strip at 29.97fps. This is a build that's based on master well after 2.78b (hash is f0cf15b), yet it exhibits the earlier behaviour. I suspect it's a difference in some library or other rather than in the Blender source code itself, as the numbers appear during calls to avformat_open_input and avformat_find_stream_info (anim_movie.c, function startffmpeg). Am I right in thinking that the blender builds on blender.org bundle libavformat etc. statically linked? ldd doesn't show any links to it, whereas it certainly does on the one I've built from source.

Olly Funkster commented

2017-04-11 11:50:43 +02:00

Also N.B. the output of ffprobe for the "bad" clip is different from that quoted in the original report:

Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

Note this shows tbr (ffmpeg's interpretation of frame rate) as 29.97, whereas for the clip that has been processed by ffmpeg it shows it as 59.94. Both have tbc of 59.94.

Also N.B. the output of ffprobe for the "bad" clip is different from that quoted in the original report: Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc Note this shows tbr (ffmpeg's interpretation of frame rate) as 29.97, whereas for the clip that has been processed by ffmpeg it shows it as 59.94. Both have tbc of 59.94.

Kory Prince commented

2017-04-11 17:40:23 +02:00

You certainly know more than I, but I think you're right about it being a library difference. Here's my ffprobe output:

Original:
Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc
ffmpeg clip:
Stream #0:0[0x100]: Video: h264 (High) ([27]- [x]- [x]- [x] / 0x001B), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc
dd clip:
Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc

I pull this information from blender on the ffmpeg versions:

2.78a blender.org:
avcodec: '56, 60, 100'
avdevice: '56, 4, 100'
avformat: '56, 40, 101'
avutil: '54, 31, 100'
swscale: ' 3, 1, 101'

2.78b blender.org:
avcodec: '57, 64, 101'
avdevice: '57, 1, 100'
avformat: '57, 56, 100'
avutil: '55, 34, 100'
swscale: ' 4, 2, 100'

I'm trying now to test different versions of ffmpeg, but it's a bit harder to find older static builds.

You certainly know more than I, but I think you're right about it being a library difference. Here's my ffprobe output: **Original:** Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc **ffmpeg clip:** Stream #0:0[0x100]: Video: h264 (High) ([27]- [x]- [x]- [x] / 0x001B), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc **dd clip:** Stream #0:0[0x1011]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc I pull this information from blender on the ffmpeg versions: **2.78a blender.org:** avcodec: '56, 60, 100' avdevice: '56, 4, 100' avformat: '56, 40, 101' avutil: '54, 31, 100' swscale: ' 3, 1, 101' **2.78b blender.org:** avcodec: '57, 64, 101' avdevice: '57, 1, 100' avformat: '57, 56, 100' avutil: '55, 34, 100' swscale: ' 4, 2, 100' I'm trying now to test different versions of ffmpeg, but it's a bit harder to find older static builds.

Kory Prince commented

2017-04-11 20:32:39 +02:00

I am now fairly positive that the problem is in ffmpeg. I have submitted a bug on the ffmpeg bug tracker detailing my findings: https://trac.ffmpeg.org/ticket/6307

For reference, attached is a fresh smaller clip straight from the camera (so dd is not necessary.)

1080i_29.97.mts

I am now fairly positive that the problem is in ffmpeg. I have submitted a bug on the ffmpeg bug tracker detailing my findings: https://trac.ffmpeg.org/ticket/6307 For reference, attached is a fresh smaller clip straight from the camera (so dd is not necessary.) [1080i_29.97.mts](https://archive.blender.org/developer/F542810/1080i_29.97.mts)

Olly Funkster commented

2017-04-11 20:46:39 +02:00

Good work!

Should we mark this ticked as "invalid" as it's in an external library?

Good work! Should we mark this ticked as "invalid" as it's in an external library?

Kory Prince commented

2017-04-11 22:45:47 +02:00

@Funkster-3 Something like that. Not sure how this system works, but some can mark a bug as "upstream." I don't see any way for me to do that.

Aaron Carlisle commented

2017-04-15 21:39:58 +02:00

Added subscribers: @Sergey, @Blendify

Sergey Sharybin was assigned by Aaron Carlisle

2017-04-15 21:39:58 +02:00

Aaron Carlisle commented

2017-04-15 21:39:58 +02:00

If there were an issue, it would be a regression since 01fa4fb6 but I suspect the duplicated timebase is correct for paff-encoded video, see also tickets #3339 and #5378. In any case, there is most likely a bug in blender as FFmpeg always detects the correct length of the clip, no matter the timebase

@Sergey can you take a look and see if there is anything from our side to fix this?

> If there were an issue, it would be a regression since 01fa4fb6 but I suspect the duplicated timebase is correct for paff-encoded video, see also tickets #3339 and #5378. In any case, there is most likely a bug in blender as FFmpeg always detects the correct length of the clip, no matter the timebase @Sergey can you take a look and see if there is anything from our side to fix this?

Olly Funkster commented

2017-04-18 13:25:49 +02:00

@Blendify ffmpeg gets the length right because the duration (in frames) has doubled as well as the tbr. If you let Blender change the scene rate to whatever ffmpeg tells it, the length in elapsed time is correct.