Importing any video to VSE results in audio and video having different lengths (not a codec problem, happens to all videos) #54655
Labels
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
8 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#54655
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
System Information
Xubuntu 16.04, I am using a Dell and an Asus laptops. I don't think my graphics card matters for Blender VSE.
Blender Version
Broken: All later versions: 2.78 and 2.79 (obtained form the website) (although have not tested with 2.77)
Worked: Blender 2.76b
f337fea
(obtained form the website)Short description of error
Importing any video to Blender VSE results in different video and audio lengths, with video usually being shorter than the audio. The fps chosen matches the video fps. Same video file is imported correctly in Blender 2.76b.
In the vast majority of cases (basically, a careful way of saying always) fiddling with custom fps settings doesn't solve the problem, the difference remains. The issue was reproduced by Blender users on the #blender IRC channel.
A very important note: the same happens to all videos with constant bit rates .
This ticket is not about supporting "a codec", this ticket is about fixing a bug that has broken video import across codecs. Also, I have to repeat here, otherwise people don't read further and attempt to close the ticket:
The problem is reproducible for me in the vast majority of cases, with all sorts of video files. This includes screencasting videos made with the SimpleScreenRecorder and all videos downloaded from YouTube. In fact, I was not able to find a file for which it doesn't happen.
Exact steps for others to reproduce the error
Please note, you don't have to use my file. The problem occurs with virtually any video file that has audio and video. Download one from YouTube. This ticket is not about supporting "a codec", this is a bug across codecs.
This is the test project with a video file. The video was shot using a standard camera app of a Samsung Galaxy 4mini phone.
https://louigiverona.com/files/var/TestProject/
The video is 95 Mb (it is only 50 sec), wanted to make it somewhat longer, since for very short videos the difference might not be that large, although typically still there, even if by 1.
Since I put the file on my server, I ask to not download it if you are not planning to work on the bug, so that I don't get a huge bill by the end of the month :)
Results of the test:
Lengths in 2.76b:
audio 1404
video 1404
Lengths in 2.79:
audio 1404
video 1391
The problem is reproducible for me in the vast majority of cases, with all sorts of video files. This includes screencasting videos made with the SimpleScreenRecorder and videos downloaded from YouTube. The only reason I am including a sample file is because clear reproduction steps are required to file a ticket of this sort. But I could have asked anyone to take any file that has audio and video and import it into Blender VSE, with same results.
All videos from any of the cameras I own, from three mobile phones (Samsung Galaxy 4mini, Huawei P8 Lite, Samsung A3) and from an expensive Nikon camera - all videos without exception always end up being opened with different lengths in 2.78 and 2.79. At the same time, they are all in complete sync in Blender 2.76b, with audio and video always having the exact same length.
I have asked several people to reproduce the issue in Blender 2.79 and all the people I asked were able to reproduce the problem immediately, using their own video files.
Another way to reproduce is to download a video from YouTube. Here is a random video I watched, downloaded and tested: https://www.youtube.com/watch?v=FNLoxaSSQ2A
Here, inspite of the video being long, the difference is smaller - but still there. The video is at 29.970030 frame rate.
Blender 2.76b:
audio 200391
video 200391
Blender 2.79:
audio 200391
video 200389
It seems that the ability of later versions of Blender to automatically detect the imported video fps is at play here. Many videos will not have exactly 30 fps, for example the frame rate of the video I supply with my test project is 30.003730. This difference is small enough to be neglected, but as JA12 in the IRC channel found out, Blender seems to try to convert this fps to some exact number and fails because of the length of the field, however, he was not certain this is really the cause. He was able to immediately reproduce an issue with my project and by importing some other video and seeing the same problem. Everyone on IRC were able to reproduce the problem using my TestProject file with the video also. JA12 was also able to reproduce the problem with the latest development version of Blender.
I will do my best to provide additional information is necessary.
Added subscriber: @LouigiVerona
Added subscriber: @ChristopherAnderssarian
Having a look at the metadata; the audio is longer than the video:
Video: Line 23: 46 s 361 ms
Audio: Line 50: 46 s 784 ms
Blender seems to be setting the correct frame rate there's just a bit of extra audio.
P658: Metadata inspection of #54655's video sample
I am not sure how is this possible, unless this is really how all cameras and videos in general work. And somehow Blender 2.76b does everything correctly, whereas the "correct" frame rate of later versions results in audio and video being out of sync.
Added subscriber: @Paskperfect
Yes, this in general is how all cameras and video recorders work, these days. Why? Gone are the days when you needed a bit of magnetic tape to move across a tape-head at a constant rate, or a constant 33 1/3 rotations per minute to faithfully reproduce video or sound.
All readily-available video and audio is compressed in some form, to either make it fit into a smaller distribution medium, or stream "faster" through phone lines, to save someone time, or money.
Before you load a file for editing in Blender, consider using a program (such as Handbrake) that can convert between variable-rate and constant-rate frames-per-second formats. You might also try FFMPeg, which is used by both Blender and Handbrake to do their video conversions for exporting to different formats. Handbrake is a lot easier to work with, than FFMPeg, though.
Other editing suites do similar conversions, without telling you about it, and still take just as long to process.
Stephen, this is all fine, but somehow Blender 2.76b was doing it correctly, without requiring me to use Handbrake or any other converter.
"Other editing suites do similar conversions, without telling you about it, and still take just as long to process."
Well, I tried Kdenlive. I imported the same file - there was no processing of any sort, it just got added to the timeline, everything perfectly in sync.
Added subscriber: @Funkster-3
Since the audio length in your reports is the same between 2.76b and later versions, and it's the video that has changed length, I would argue that 2.76b was getting the video length wrong somehow. It may even be a difference in ffmpeg / libavcodec rather than in blender itself, since the library versions also change between blender releases.
FWIW, my local builds are based around 2.78ish, and I get matching audio and video lengths with the cameras I use:
None of these cameras use variable frame rate. No editing system can perfectly match audio and video if the video frame interval is not consistent. Mobile phones and OBS are notorious for creating videos with inconsistent frame intervals due to them being busy doing other things with the CPU.
I'm not at my main machine right now, next time I am I'll try some clips in the 2.78 and 2.79 releases but I don't recall having any problems.
Okay, so I just took the test video which I supplied, used Handbrake to convert it to 30 constant fps. Now Codec info says exactly 30 fps.
Opening it in Blender 2.79:
audio: 1405
video: 1391
Project fps is at 30.
Opening it in Blender 2.76b:
audio: 1405
video: 1405
Project fps is at 30.
So, even with the constant frame rate the problem seems to persist.
@LouigiVerona Try importing your converted video in 2.79b (or preferably a daily build ) and see if the problem persists.
I just downloaded your sample... it only contains 1391 video frames (checked by transcoding it with ffmpeg and looking at the frame count at the end).
So any setup that makes you a video strip 1405 frames long is adding some frames somewhere to compensate.
Added subscriber: @TobiaszunfaKaron
Recently I had a problem with audio exported from Ardour being longer than the same audio from OBS video file (both at 48kHz sampling rate both PCM streams).
It broke both in Blender and Kdenlive in similar ways. I think it could be a problem in one of the libraries these programs use to parse audio stream metadata or something.
Not sure if it's related though.
@Funkster-3 - Could your result be because ffmpeg is not compensating correctly for some dropped frames in the stream?
If the same file plays fine in VLC or another media player I guess it's a problem in Blender or some libraries Blender use. I know that editing video is more complex than simply playing it back, but I don't see a reason why it should make the A/V sync break. It's not what I'd expect to happen in a video editor.
@TobiaszunfaKaron VLC is pretty clever, and if the file contains information about a dropped frame or an altered presentation time, you can bet that VLC is capable of interpreting that correctly and adjusting the timing to stay in sync. Blender just takes the average frame rate, and the index of the frame you're on, and calls it a day.
Hmm, maybe that's the root of the problem? But in such case - could transcoding (or just remuxing) the file to a constant FPS file could help?
@Funkster-3 If you read the info I have provided above, this doesn't add up. Blender does everything correctly in previous versions, and so do Kdenlive and even OpenShot.
My forced workaround is to use an old version of Blender for editing, but then render it through the new version of Blender. Basically, I can just import all the files through the old version of Blender I mentioned, and once they are imported, this new Blender version can open the files normally and work with them.
So, the problem is very clearly in the change of file importing into VSE. I am confused as to why this demonstrable fact is still being disputed.
Any change in behaviour is in FFMPEG, not in the VSE code itself (which just takes the frames and the playback rate data from ffmpeg as it has always done).
But the real source of the error is your video file, whose length, frame count, and playback rate do not add up, and that is not blender's responsibility to fix - given the extremely limited (i.e. near-zero) amount of coding resource available for VSE development, dealing more gracefully with non-compliant video files is about as near to the bottom of the priority list as it's possible to get.
If you are able to develop a fix for your edge case that doesn't break anything else, great. Otherwise I would suggest that some kind of transcoding / editing of frame rate data / change to the way your source video is generated is your only option at this point.
Good luck! :o)
I respectfully disagree. Please, see comment https://developer.blender.org/T54655#494818 where I have transcoded a file to have audio of exact same length as video, and the import into Blender still failed. And yet it works completely fine in an old version.
Added subscriber: @ddilshod
I have the same problem. My workaround was to change the frame rate of my galaxy note 9 to 60 when taking videos in full hd. Then I adjusted the custom rate so it matches the frame rate of 59.06 because the windows file properties shows this rate. Didn't even bother to see the real precise frame rate in VLC because the number would be somewhat of 59.060004001. This setup gave me a little longer audio than the video. The difference is so minor that even in long videos it's unnoticeable, I just cut the end of the audio so it matches the video. To me it's clearly a bug. It's supposed to identify the floating frame rate just like in Shotcut, there the videos are perfect, in Blender for some reason you have to work around it....
Added subscriber: @iss
Changed status from 'Open' to: 'Archived'
I am once again looking at this task, and see, that the sample video has variable frame rate.
We can not support every single video codec. We do support most "mainstream" codecs. I am not aware about any demand for supporting variable frame rate codecs in Blender.
PS: You say that Blender version 2.76b worked. It may seem, like it did, but I doubt, that file was decoded properly.
PPS: Even if Blender would support this codec, you would probably want to make proxies for this footage. You can use indexed codec when converting from VFR to CFR and skip proxy building in Blender.
Changed status from 'Archived' to: 'Open'
Dear Richard,
This ticket is not at all about supporting "a codec". If you would, in fact, look at this comment , you would see that videos with constant bit rate have exactly the same problem. This is the second time I have to say this, because people fail to read the discussion. I will now edit this information into the original description so that this doesn't happen again.
Moreover, I have tried importing videos from various devices, random videos downloaded from YouTube - and each and every one of them has this problem. And none of them have this problem in 2.76b. In fact, I was not able to find a video which doesn't have this problem, and everyone else whom I asked to check on their machines have been able to duplicate the problem.
So, this is not a ticket about supporting a codec, this is a ticket describing a bug which makes using Blender VSE for imported videos (any videos with video+audio, any codec) impossible after version 2.76b.
Importing any video to VSE results in audio and video having different lengthsto Importing any video to VSE results in audio and video having different lengths (not a codec problem, happens to all videos)@LouigiVerona Sorry for misunderstanding.
While it's true, that I constantly overlook details, but still sometimes, less is more(description is quite verbose).
Anyway, I reviewed 4 files:
Following videos did have small difference(<5frames) in audio and video length - seems good to me.
Most video containers do support storage of audio with length independent to video length. However all these cases, Blender is not responsible for this discrepancy.
Practically speaking, Blender can trim content of audio or video, to be aligned, however this is not really nice thing to do.
Changed status from 'Open' to: 'Archived'
Closing this again.
I also tested this in 2.76 - 1405 frames imported, but after frame 1391 movie strip output was nothing.
This means, that your file was probably recoded incorrectly, and that 2.76 would also be out of sync.
If you can demonstrate this problem with another file, please create new task for this.
Added subscriber: @Fabien-Marello
Just adding that the workaround described earlier worked for me :
I converted my file from a "Peak Framerate" mp4 to a "Constant Framerate" m4v with Handbrake. My frame number wasn't divided by nearly 2 anymore (which is not exactly the described problem, that occurs on smaller scales, but it was the single related topic that I found).
Thanks.