VSE: Frame Rate Base Rounding Leads To Vid / Audio Strips Out Of Sync by 1 Frame #110878

New Issue

Closed

opened 2023-08-07 09:41:23 +02:00 by Daniel · 10 comments

Daniel commented

2023-08-07 09:41:23 +02:00

System Information
Operating system: Windows 10
Graphics card: RTX 3090

Blender Version
Broken: since 3.0 until now in 3.6.1 hash 8bda729ef4
Worked: 2.93.18

Short description of error
It looks like the same bug #21138 is reappearing in the 3.x series and since the bug report is 14 years old, I created a new one.
Old bug report: #21138

My description:
I have a lot of projects with 29.97fps (30000/1001) and those work correctly with Blender 2.9x but since Blender 3.0 all video clips are 1 frame shorter and my projects are a mess, because the frames in all clips are also shifted 1 frame !!
If I set the base to 1.0009 instead of 1.001 its all fine in the Timeline, but if I set 29.97 from the drop-down menu, all clips are 1 frame shorter and shifted.
I have this also tested with videos without any audio and that makes no difference.
The audio length btw is alway correct.
I also noticed that the trick with 1.0009 is not good, because this value is then transferred to the exported video and thats wrong!

I think the problems in the following posts also have the same cause?
https://blender.stackexchange.com/questions/51650/audio-is-1-frame-off
https://blender.stackexchange.com/questions/250718/blender-3-0-vse-audio-and-video-out-of-sync-bug

**System Information** Operating system: Windows 10 Graphics card: RTX 3090 **Blender Version** Broken: since 3.0 until now in 3.6.1 hash 8bda729ef4dc Worked: 2.93.18 **Short description of error** It looks like the same bug #21138 is reappearing in the 3.x series and since the bug report is 14 years old, I created a new one. Old bug report: https://projects.blender.org/blender/blender/issues/21138 My description: I have a lot of projects with 29.97fps (30000/1001) and those work correctly with Blender 2.9x but since Blender 3.0 all video clips are 1 frame shorter and my projects are a mess, because the frames in all clips are also shifted 1 frame !! If I set the base to 1.0009 instead of 1.001 its all fine in the Timeline, but if I set 29.97 from the drop-down menu, all clips are 1 frame shorter and shifted. I have this also tested with videos without any audio and that makes no difference. The audio length btw is alway correct. I also noticed that the trick with 1.0009 is not good, because this value is then transferred to the exported video and thats wrong! I think the problems in the following posts also have the same cause? [https://blender.stackexchange.com/questions/51650/audio-is-1-frame-off](https://blender.stackexchange.com/questions/51650/audio-is-1-frame-off) [https://blender.stackexchange.com/questions/250718/blender-3-0-vse-audio-and-video-out-of-sync-bug](https://blender.stackexchange.com/questions/250718/blender-3-0-vse-audio-and-video-out-of-sync-bug)

Daniel added the

labels 2023-08-07 09:41:24 +02:00

Iliya Katushenock added the

Video Sequencer

label 2023-08-07 12:00:23 +02:00

YimingWu commented

2023-08-08 04:18:54 +02:00

Member

I think a new issue is fine.

Since the intention of 29.97 is 30 / 1.001 and it's a little bit slightly over 29.97 (29.97002997), I'm not sure if we add the value just ever so slightly so that the audio strip will visually align with video length in such scenarios.

I think a new issue is fine. Since the intention of 29.97 is `30 / 1.001` and it's _a little bit slightly_ over 29.97 (29.97002997), I'm not sure if we add the value just ever so slightly so that the audio strip will visually align with video length in such scenarios.

YimingWu added

and removed

labels 2023-08-08 04:20:45 +02:00

Daniel commented

2023-08-08 13:10:15 +02:00

Author

I think the main problem is also the shift of 1 frame so the first frame is missing and thats very bad for compositing. very strange.
Blender 2.93.18 works fine

I think the main problem is also the shift of 1 frame so the first frame is missing and thats very bad for compositing. very strange. Blender 2.93.18 works fine

Richard Antalik commented

2023-08-11 07:09:49 +02:00

Member

Can you clarify whether the issue is, that imported movie is 1 frame shorter than expected, or the issue is, that older files are now somewhat offset?

As far as AV sync goes, alignment should be perfect. If it is no, that can be bug. It is likely, that I will need a sample of movie to reproduce this issue. Can you upload it with .blend file?

Can you clarify whether the issue is, that imported movie is 1 frame shorter than expected, or the issue is, that older files are now somewhat offset? As far as AV sync goes, alignment should be perfect. If it is no, that can be bug. It is likely, that I will need a sample of movie to reproduce this issue. Can you upload it with .blend file?

Daniel commented

2023-08-11 21:57:49 +02:00

Author

I have now invested a few hours to reproduce the bugs, which is not easy. I have put here a few examples to download and below I try to describe the sample files. Download

blend-files are created with Blender 3.6.1 LTS
All video-clips has 20 frames with a number in it from 1 to 20
There is no audio in the clips because I think that they are all played correctly.

video-clips with 'exact' has for example 30000/1001 ---> this is the standard-format in NTSC
video-clips without 'exact' has for example 29.97 ---> this is none standard-format in NTSC but files exists in the internet

files are created with ffmpeg for example with:
ffmpeg.exe -f image2 -framerate 30000/1001 -i "%1%%04d.tiff" -c:v png -pred sub "%1_exact.mov"
ffmpeg.exe -f image2 -framerate 29.97 -i "%1%%04d.tiff" -c:v png -pred sub "%1.mov"

frameshift in the timeline mostly occurs with framerate 29.97 / 59.94 / 23,976 / 14.985
clip in the timeline is mostly one frame shorter with 30000/1001 ; 60000/1001 ; 15000/1001 ; ( but 24000/1001 does not???)
If both kinds of files are in the timeline, accurate work is not possible in Blender 3.x
If I recreate all blend-files in Blender 2.93.18 or open my old files in 2.93, none of this problems are exists.

First I noticed this bug, when I opened the old project files from 2.93 with Blender 3.0 to 3.6.1 and I quickly saw that there was a frame gap after each clip with flicker at the transition.
After that, I also noticed that in my timeline are files with 29.97 and 30000/1001 at the same time, which I never noticed before.
So I guess that the frameshifts comes from that bad files with 29.97 and the shorter clips from 30000/1001

It's also important to know, that it depends which kind of file you put at first in the timline.
It is also strange that if you put only correct files with 30000/1001 in the timeline, at first it looks all ok, but even in the drop-down-list is 29.97 selected and you select the same accidentally again, all clips in the timeline are then 1 frame shorter, and you have no possibility to undo this misstake.(Maybe there is a solution, but I don't know it.)

I have now invested a few hours to reproduce the bugs, which is not easy. I have put here a few examples to download and below I try to describe the sample files. [Download](https://f003.backblazeb2.com/file/cd1188-open/blender/sample_VSE_BUG-110878.zip) blend-files are created with Blender 3.6.1 LTS All video-clips has 20 frames with a number in it from 1 to 20 There is no audio in the clips because I think that they are all played correctly. video-clips with 'exact' has for example 30000/1001 ---> this is the standard-format in NTSC video-clips without 'exact' has for example 29.97 ---> this is none standard-format in NTSC but files exists in the internet files are created with ffmpeg for example with: ffmpeg.exe -f image2 -framerate 30000/1001 -i "%1\%%04d.tiff" -c:v png -pred sub "%1_exact.mov" ffmpeg.exe -f image2 -framerate 29.97 -i "%1\%%04d.tiff" -c:v png -pred sub "%1.mov" ------------------------------------------------------------------------------------------ frameshift in the timeline mostly occurs with framerate 29.97 / 59.94 / 23,976 / 14.985 clip in the timeline is mostly one frame shorter with 30000/1001 ; 60000/1001 ; 15000/1001 ; ( but 24000/1001 does not???) If both kinds of files are in the timeline, accurate work is not possible in Blender 3.x If I recreate all blend-files in Blender 2.93.18 or open my old files in 2.93, none of this problems are exists. First I noticed this bug, when I opened the old project files from 2.93 with Blender 3.0 to 3.6.1 and I quickly saw that there was a frame gap after each clip with flicker at the transition. After that, I also noticed that in my timeline are files with 29.97 and 30000/1001 at the same time, which I never noticed before. So I guess that the frameshifts comes from that bad files with 29.97 and the shorter clips from 30000/1001 ------------------------------------------------------------------------------------------ It's also important to know, that it depends which kind of file you put at first in the timline. It is also strange that if you put only correct files with 30000/1001 in the timeline, at first it looks all ok, but even in the drop-down-list is 29.97 selected and you select the same accidentally again, all clips in the timeline are then 1 frame shorter, and you have no possibility to undo this misstake.(Maybe there is a solution, but I don't know it.)

Richard Antalik commented

2023-08-14 07:28:49 +02:00

Member

From what you describe, this seems to be due to FPS mismatch, even though this looks like a bug in current implementation.

If scene FPS is different than movie FPS, and "Adjust Playback Rate" option is enabled when adding movie strip, it will drop or repeat frames as needed to match scene playback rate. I guess, that is what is happening here.
Also there is option for setting scene FPS to be same as first added file, so this may have confused you even more.

So I can look at the algorithm how frame dropping is calculated - I think, that it does floor() conversion so frame 1.999999 is displayed as frame 1. Not sure if this is best solution from AV sync point of view, probably not, but I have tried to replicate ffmpeg seeking behavior, which may also be improved in this regard...

And another issue is strip length - question is: Is it OK to display strip of length 19.9999 as 20 frames? There would have to be threshold where this would cause problems, and this is a bit of edge case.

Regardless of whether I can find solution, it should be possible to transcode files to common framerate with ffmpeg. Blender calculates number of frames from FPS and movie duration in seconds, so this may also return "incorrect" number, making things even worse

From what you describe, this seems to be due to FPS mismatch, even though this looks like a bug in current implementation. If scene FPS is different than movie FPS, and "Adjust Playback Rate" option is enabled when adding movie strip, it will drop or repeat frames as needed to match scene playback rate. I guess, that is what is happening here. Also there is option for setting scene FPS to be same as first added file, so this may have confused you even more. So I can look at the algorithm how frame dropping is calculated - I think, that it does `floor()` conversion so frame 1.999999 is displayed as frame 1. Not sure if this is best solution from AV sync point of view, probably not, but I have tried to replicate ffmpeg seeking behavior, which may also be improved in this regard... And another issue is strip length - question is: Is it OK to display strip of length 19.9999 as 20 frames? There would have to be threshold where this would cause problems, and this is a bit of edge case. Regardless of whether I can find solution, it should be possible to transcode files to common framerate with ffmpeg. Blender calculates number of frames from FPS and movie duration in seconds, so this may also return "incorrect" number, making things even worse

Germano Cavalcante referenced this issue

2023-09-02 00:38:15 +02:00

VSE: When changing the 'Speed Factor' of an audio strip, the 'Strip Offset' values are not automatically updated #111797

Richard Antalik referenced a pull request that will close this issue

2023-10-18 03:00:34 +02:00

Fix #110878 VSE duplicating first frame and offsetting rest. #113870

Richard Antalik referenced this issue from a commit

2023-10-19 02:08:29 +02:00

Fix #110878 VSE duplicating first frame and offsetting rest.

Richard Antalik closed this issue

2023-10-19 02:08:30 +02:00

Blender Bot added

and removed

labels 2023-10-19 02:08:32 +02:00

Daniel commented

2023-10-22 13:37:59 +02:00

Author

@iss I have just checked the new Beta 4.0 (2023-10-21 / Hash: 2e084addb5) with the bugfix and the frameshift is now gone, but I noticed that for example the test file test_29.97_exact.mov is still missing the last frame and is showing only 19 frames instead of 20.
This happens If I set the scene framerate to 29.97 from the pulldown menu in the gui. so the bugfix is not pefect.

Another example
If I have several clips in a row and then change scene-framerate, there is a gap of 1 frame between the clips and a black flicker at the transition.

I'm still wondering why this is only a problem with file-framerate 30000/1001 and 60000/1001 but not 24000/1001 ?
Is there maybe a special case with 23.976 fps in the source code?

@iss I have just checked the new Beta 4.0 (2023-10-21 / Hash: 2e084addb542) with the bugfix and the frameshift is now gone, but I noticed that for example the test file test_29.97_exact.mov is still missing the last frame and is showing only 19 frames instead of 20. This happens If I set the scene framerate to 29.97 from the pulldown menu in the gui. so the bugfix is not pefect. Another example If I have several clips in a row and then change scene-framerate, there is a gap of 1 frame between the clips and a black flicker at the transition. I'm still wondering why this is only a problem with file-framerate 30000/1001 and 60000/1001 but not 24000/1001 ? Is there maybe a special case with 23.976 fps in the source code?

Daniel commented

2023-10-22 14:57:25 +02:00

Author

@iss @Sergey Maybe I'm getting closer to the problem..
When I drag the first video file to the timeline, the scene frame rate changes. In the screenshot you can see that only the file 24000/1001 is recognized correctly and therefore this frame rate has no problems. Looks like the cause could be the detection of the frame rate of the individual clips?

@iss @Sergey Maybe I'm getting closer to the problem.. When I drag the first video file to the timeline, the scene frame rate changes. In the screenshot you can see that only the file 24000/1001 is recognized correctly and therefore this frame rate has no problems. Looks like the cause could be the detection of the frame rate of the individual clips? ![grafik](/attachments/c939077d-04ff-431c-b978-beda0ce2271b)

14 KiB

Richard Antalik commented

2023-10-23 04:35:02 +02:00

Member

@salvation Yes, the frame rate of scene is changed with first movie you add to VSE. VSE will know, that there are 20 frames in the movie with slightly different FPS. But it calculates the length with framerate difference in mind and this will result in strip length of 19.9999786. So the value is simply truncated to lower integer. This is lazy, but safe. Reasoning for this implementation was, that nobody would likely notice, when it comes to editing movies, but that assumes sources like camera where you trim footage quite a bit anyway.

Currently, when you move a strip handle to make it 20 frames long, the missing frame will still be rendered.

I will have to think about how I can resolve this in a way that would ensure, that all possible frames are visible, but at the same time I can not allow last frame to be not renderable.

@salvation Yes, the frame rate of scene is changed with first movie you add to VSE. VSE will know, that there are 20 frames in the movie with slightly different FPS. But it calculates the length with framerate difference in mind and this will result in strip length of 19.9999786. So the value is simply truncated to lower integer. This is lazy, but safe. Reasoning for this implementation was, that nobody would likely notice, when it comes to editing movies, but that assumes sources like camera where you trim footage quite a bit anyway. Currently, when you move a strip handle to make it 20 frames long, the missing frame will still be rendered. I will have to think about how I can resolve this in a way that would ensure, that all possible frames are visible, but at the same time I can not allow last frame to be not renderable.

Richard Antalik commented

2023-10-23 04:37:24 +02:00

Member

@salvation Can you make another report for length issue in case I won't be able to resolve it immediately, so I won't forget?
I have done some experimentation and found, that in some cases rounding does result in a empty frame at the end of the video. So to resolve this, I need to check if last frame index multiplied by playback factor is less than content length after rounding. I guess I could wrap this around and check from the other end to avoid condition, but still...

@salvation Can you make another report for length issue in case I won't be able to resolve it immediately, so I won't forget? I have done some experimentation and found, that in some cases rounding does result in a empty frame at the end of the video. So to resolve this, I need to check if last frame index multiplied by playback factor is less than content length after rounding. I guess I could wrap this around and check from the other end to avoid condition, but still...

Daniel referenced this issue

2023-10-24 21:05:21 +02:00

VSE: Movie strip with NTSC-framerate are missing the last frame in the timeline #114122

Daniel commented

2023-10-24 21:07:37 +02:00

Author

@iss I have opend a new issue report.

@iss I have opend a new issue report.

Sign in to join this conversation.

No Label

Animation & Rigging

Asset Browser Project Overview

Automated Testing

Blender Asset Bundle

Dependency Graph

Development Management

EEVEE & Viewport

Images & Movies

Motion Tracking

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Render Pipeline

Sculpt, Paint & Texture

Video Sequencer

Virtual Reality

Blender 2.8 Project

Milestone 1: Basic, Local Asset Browser

Good First Issue

Animation & Rigging

Development Management

EEVEE & Viewport

Nodes & Physics

Pipeline, Assets & IO

Platforms, Builds & Tests

Render & Cycles

Sculpt, Paint & Texture

Needs Info from Developers

Needs Information from User

No Milestone

No project

No Assignees

3 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#110878

No description provided.