VSE: Very slow export time of retimed video strips #126472

Closed
opened 2024-08-18 22:45:39 +02:00 by ThinkingPolygons · 17 comments

System Information
Operating system: Windows 10

Blender Version
Broken: blender-4.3.0-alpha+main.56779c7bb05f-windows.amd64-release
Worked: unsure, maybe it was always like that

I can't share the footage, so to reproduce you must have at least a 1080p video of around 20-30 minutes.

Let's go:
File/New/Video Editing
Load the video in the VSE
With the video strip selected, in the sidebar/Strip tab, open the Time panel and check the "Show Retiming Keys" checkbox
Now click on any of the retiming keyframes that appeared on the strip
Press R, a popup to set the speed will appear
Type 800 in the "Speed" number field and hit enter
The strip should now be quite short to match the new speed
If you press play, you can see that the video is now playing 800% faster

Now it's time to export the video:
Adjust the start/end frame range to match the strip's duration
The Render resolution must be adjusted to match the video's resolution if needed
In the properties editor/Output tab/Output panel, set the file output path to wherever you want
Expand the Encoding subpanel, leave everything as it is, just set the Audio Codec to None (I tested on a video without audio channel)
Go to the Render Menu and click Render Animation (Ctrl+F12)

I hope you can see that the export time is much much slower than usual. In the render window you can see that the rendered frames doesn't even match the playback speed.

In comparison, this same operation in the open source video editor Shotcut it exports very fast.

**System Information** Operating system: Windows 10 **Blender Version** Broken: blender-4.3.0-alpha+main.56779c7bb05f-windows.amd64-release Worked: unsure, maybe it was always like that I can't share the footage, so to reproduce you must have at least a 1080p video of around 20-30 minutes. Let's go: File/New/Video Editing Load the video in the VSE With the video strip selected, in the sidebar/Strip tab, open the Time panel and check the "Show Retiming Keys" checkbox Now click on any of the retiming keyframes that appeared on the strip Press R, a popup to set the speed will appear Type 800 in the "Speed" number field and hit enter The strip should now be quite short to match the new speed If you press play, you can see that the video is now playing 800% faster Now it's time to export the video: Adjust the start/end frame range to match the strip's duration The Render resolution must be adjusted to match the video's resolution if needed In the properties editor/Output tab/Output panel, set the file output path to wherever you want Expand the Encoding subpanel, leave everything as it is, just set the Audio Codec to None (I tested on a video without audio channel) Go to the Render Menu and click Render Animation (Ctrl+F12) I hope you can see that the export time is much much slower than usual. In the render window you can see that the rendered frames doesn't even match the playback speed. In comparison, this same operation in the open source video editor [Shotcut](https://www.shotcut.org) it exports very fast.
ThinkingPolygons added the
Severity
Normal
Status
Needs Triage
Type
Bug
labels 2024-08-18 22:45:39 +02:00
Member

I believe VSE is layering frames together to form a new frame under this kind of situations...

In seq_render_effect_strip_impl we do have a 3 times loop in the DoEffect part (Not sure if this is the correct place)? poke @iss

I believe VSE is layering frames together to form a new frame under this kind of situations... In `seq_render_effect_strip_impl` we do have a 3 times loop in the `DoEffect` part (Not sure if this is the correct place)? poke @iss

@ChengduLittleA seq_render_effect_strip_impl is probably not called at all with retimed strips, and with speed effect, the loop iterates only once, because there is only 1 input.

I can't speculate, on what is the reason. I can reproduce about 25% slower rendering compared to strip, that is not retimed.
Strip length does not seem to have any influence on rendering performance when tested here.

@ThinkingPolygons How much slower the rendering speed when compared same strip with and without retiming? In any case, I would need .blend file and the movie file you are using to reproduce this it seems.

@ChengduLittleA `seq_render_effect_strip_impl` is probably not called at all with retimed strips, and with speed effect, the loop iterates only once, because there is only 1 input. I can't speculate, on what is the reason. I can reproduce about 25% slower rendering compared to strip, that is not retimed. Strip length does not seem to have any influence on rendering performance when tested here. @ThinkingPolygons How much slower the rendering speed when compared same strip with and without retiming? In any case, I would need .blend file and the movie file you are using to reproduce this it seems.
Richard Antalik added
Status
Needs Information from User
and removed
Status
Needs Info from Developers
labels 2024-08-19 12:55:15 +02:00

I can't tell exactly how slower it is compared to a non-retimed video, but it's unexpectedly slower.

Unfortunately I can't share the footage I used, but the file is an ordinary mp4 file, so I believe it can be reproduced with most video files out there.
I don't think it matters but here's the mediainfo of the file, in case you are curious:

General
Complete name                  : D:\3qEYVUdqNu.mp4
Format                         : MPEG-4
Format profile                 : Base Media
Codec ID                       : isom (isom/iso2/avc1/mp41)
File size                      : 590 MiB
Duration                       : 24 min 10 s
Overall bit rate               : 3 413 kb/s
Writing application            : Lavf61.1.100

Video
ID                             : 1
Format                         : AVC
Format/Info                    : Advanced Video Codec
Format profile                 : Baseline@L4
Format settings                : 1 Ref Frames
Format settings, CABAC         : No
Format settings, ReFrames      : 1 frame
Codec ID                       : avc1
Codec ID/Info                  : Advanced Video Coding
Duration                       : 24 min 10 s
Bit rate                       : 3 411 kb/s
Width                          : 1 920 pixels
Height                         : 1 016 pixels
Display aspect ratio           : 1.890
Frame rate mode                : Constant
Frame rate                     : 30.000 FPS
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Bits/(Pixel*Frame)             : 0.058
Stream size                    : 590 MiB (100%)
Writing library                : x264 core 164
Encoding settings              : cabac=0 / ref=1 / deblock=0:0:0 / analyse=0:0 / me=dia / subme=0 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=8 / lookahead_threads=8 / sliced_threads=1 / slices=8 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=250 / keyint_min=25 / scenecut=0 / intra_refresh=0 / rc=crf / mbtree=0 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=0
Codec configuration box        : avcC

But I'm really wondering how blender actually does this operation, because it's totally unexpected.
Does blender renders retimed video strips as if they weren't retimed at all behind the scenes?
Is it expected the render time to be same or worse than non-retimed strips?
Because that's not how it seems to work at all in other video editors. In shotcut for example, after changing the speed, it renders it as if it was really just a short piece of video. So something that would take minutes to render in full length, comes out in seconds, because it's a short clip now. So something is really different in blender.

I know it's not practical but if you guys could discover how they do it in Shotcut it would be amazing. It's all ffmpeg, right?

I can't tell exactly how slower it is compared to a non-retimed video, but it's unexpectedly slower. Unfortunately I can't share the footage I used, but the file is an ordinary mp4 file, so I believe it can be reproduced with most video files out there. I don't think it matters but here's the mediainfo of the file, in case you are curious: ``` General Complete name : D:\3qEYVUdqNu.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/avc1/mp41) File size : 590 MiB Duration : 24 min 10 s Overall bit rate : 3 413 kb/s Writing application : Lavf61.1.100 Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : Baseline@L4 Format settings : 1 Ref Frames Format settings, CABAC : No Format settings, ReFrames : 1 frame Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 24 min 10 s Bit rate : 3 411 kb/s Width : 1 920 pixels Height : 1 016 pixels Display aspect ratio : 1.890 Frame rate mode : Constant Frame rate : 30.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.058 Stream size : 590 MiB (100%) Writing library : x264 core 164 Encoding settings : cabac=0 / ref=1 / deblock=0:0:0 / analyse=0:0 / me=dia / subme=0 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=8 / lookahead_threads=8 / sliced_threads=1 / slices=8 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=250 / keyint_min=25 / scenecut=0 / intra_refresh=0 / rc=crf / mbtree=0 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=0 Codec configuration box : avcC ``` But I'm really wondering how blender actually does this operation, because it's totally unexpected. Does blender renders retimed video strips as if they weren't retimed at all behind the scenes? Is it expected the render time to be same or worse than non-retimed strips? Because that's not how it seems to work at all in other video editors. In shotcut for example, after changing the speed, it renders it as if it was really just a short piece of video. So something that would take minutes to render in full length, comes out in seconds, because it's a short clip now. So something is really different in blender. I know it's not practical but if you guys could discover how they do it in Shotcut it would be amazing. It's all ffmpeg, right?

I can't tell exactly how slower it is compared to a non-retimed video

Why? You can just do it and measure time...

But I'm really wondering how blender actually does this operation, because it's totally unexpected.
Does blender renders retimed video strips as if they weren't retimed at all behind the scenes?
Not sure what do you mean. If video has 800% speed, it renders frames 0, 8, 16, ... There isn't much more to it, just a bit of trivial math.

I have noticed, that in 4.3 R key fails to adjust strip length, so

The strip should now be quite short to match the new speed

Is not true anymore.
I guess that could be the issue. Can you check?

> I can't tell exactly how slower it is compared to a non-retimed video Why? You can just do it and measure time... > But I'm really wondering how blender actually does this operation, because it's totally unexpected. > Does blender renders retimed video strips as if they weren't retimed at all behind the scenes? Not sure what do you mean. If video has 800% speed, it renders frames 0, 8, 16, ... There isn't much more to it, just a bit of trivial math. I have noticed, that in 4.3 R key fails to adjust strip length, so > The strip should now be quite short to match the new speed Is not true anymore. I guess that could be the issue. Can you check?
Contributor

Currently rendering a video with 4K footage and retimed sections.

Normal footage renders about 2 frames/sec at about 30% CPU
Retimed footage shoots CPU to 100% and takes 2..3 sec per frame

Currently rendering a video with 4K footage and retimed sections. Normal footage renders about 2 frames/sec at about 30% CPU Retimed footage shoots CPU to 100% and takes 2..3 sec per frame

Why? You can just do it and measure time...

I couldn't do it because it takes too long to render, and unfortunately at the moment I don't have enough free time to do deep testing.

Not sure what do you mean. If video has 800% speed, it renders frames 0, 8, 16, ... There isn't much more to it, just a bit of trivial math.

Right, but then why is it so slow?
Is there a possibility of a bug that makes blender somehow still render all frames behind the scenes? I don't know man, it's just really strange. This operation should be several times faster.

I have noticed, that in 4.3 R key fails to adjust strip length, so Is not true anymore.
I guess that could be the issue. Can you check?

Is that a bug? I hope so.
And I noticed that too, that's why I asked in the chat if there is a new way of doing it.
But I managed to do it with the steps in the first post (sometimes it fails), and the playback was fine, so I thought it must be working as expected. Perhaps it's not really doing it properly? Interesting.

See how I did it:


@Ha_Lo Thanks for sharing that comparison. And yeah, I was surprised at the 100% CPU as well. VSE hardly goes there.

> Why? You can just do it and measure time... I couldn't do it because it takes too long to render, and unfortunately at the moment I don't have enough free time to do deep testing. > Not sure what do you mean. If video has 800% speed, it renders frames 0, 8, 16, ... There isn't much more to it, just a bit of trivial math. Right, but then why is it so slow? Is there a possibility of a bug that makes blender somehow still render all frames behind the scenes? I don't know man, it's just really strange. This operation should be several times faster. > I have noticed, that in 4.3 R key fails to adjust strip length, so Is not true anymore. > I guess that could be the issue. Can you check? Is that a bug? I hope so. And I noticed that too, that's why I asked in the [chat](https://blender.chat/channel/sequencer-module?msg=p8t7Nm6zQjWZeyMHa) if there is a new way of doing it. But I managed to do it with the steps in the first post (sometimes it fails), and the playback was fine, so I thought it must be working as expected. Perhaps it's not really doing it properly? Interesting. See how I did it: <video src="/attachments/766ee9c4-23d1-48c5-8600-dcd14078b89a" title="blender_3ImlqXTNpi.mp4" controls></video> ___ @Ha_Lo Thanks for sharing that comparison. And yeah, I was surprised at the 100% CPU as well. VSE hardly goes there.

I am really unable to reproduce this. I attach the video:

Why? You can just do it and measure time...

I couldn't do it because it takes too long to render, and unfortunately at the moment I don't have enough free time to do deep testing.

You can just render 50 frames. If normal playback takes 2 seconds to render, then issue is not in retiming.

@Ha_Lo If you can reproduce this, can you share video file, that does this?

I am really unable to reproduce this. I attach the video: <video src="/attachments/367a8344-99d5-4a44-973c-826444bedfdd" title="2024-08-26 23-54-11.mkv" controls></video> > > Why? You can just do it and measure time... > > I couldn't do it because it takes too long to render, and unfortunately at the moment I don't have enough free time to do deep testing. You can just render 50 frames. If normal playback takes 2 seconds to render, then issue is not in retiming. @Ha_Lo If you can reproduce this, can you share video file, that does this?

Could it be a Windows only problem? 🤔

Could it be a Windows only problem? 🤔

It could. Now investigating #126826, which I can reproduce.

It could. Now investigating #126826, which I can reproduce.

I will make some assumptions and close this as duplicate of #126826, since I can reproduce that issue and likelyhood of this being duplicate is very high.

I will make some assumptions and close this as duplicate of #126826, since I can reproduce that issue and likelyhood of this being duplicate is very high.
Blender Bot added
Status
Archived
and removed
Status
Needs Information from User
labels 2024-08-29 05:15:58 +02:00

@ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed keyint=250 among encoder settings, which is like keyframes every 250 frames. Which would make "seeking" within the video potentially very slow. @ThinkingPolygons can you check on your side, whether video keyframe interval that is used on the source video file (not blender render settings!) has affect on rendering performance?

@ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed `keyint=250` among encoder settings, which is like keyframes every 250 frames. Which would make "seeking" within the video *potentially very slow*. @ThinkingPolygons can you check on your side, whether video keyframe interval that is used on the source video file (not blender render settings!) has affect on rendering performance?

@ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed keyint=250 among encoder settings, which is like keyframes every 250 frames. Which would make "seeking" within the video potentially very slow. @ThinkingPolygons can you check on your side, whether video keyframe interval that is used on the source video file (not blender render settings!) has affect on rendering performance?

interesting.
i have no idea why the encoding settings is saying that. the original video was recorded with a keyframe interval of 18. perhaps the keyint=250 means something else? 🤔

will try to check

> @ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed `keyint=250` among encoder settings, which is like keyframes every 250 frames. Which would make "seeking" within the video *potentially very slow*. @ThinkingPolygons can you check on your side, whether video keyframe interval that is used on the source video file (not blender render settings!) has affect on rendering performance? interesting. i have no idea why the encoding settings is saying that. the original video was recorded with a keyframe interval of 18. perhaps the keyint=250 means something else? 🤔 will try to check

@ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed keyint=250 among encoder settings, which is like keyframes every 250 frames.

From what I saw, yes, that would be the "issue", but it should not be the issue for rendering.

> @ThinkingPolygons @iss could this issue be caused by fairly large keyframe interval used in the video? I noticed `keyint=250` among encoder settings, which is like keyframes every 250 frames. From what I saw, yes, that would be the "issue", but it should not be the issue for rendering.

im not sure we can trust this keyint information
i checked several h264 videos from different sources and when that piece of info "keyint=250 / keyint_min=25" is available it always has those same values.
perhaps its a ffmpeg specific thing. a type of calculation. i dont know

im not sure we can trust this `keyint` information i checked several h264 videos from different sources and when that piece of info `"keyint=250 / keyint_min=25"` is available it always has those same values. perhaps its a ffmpeg specific thing. a type of calculation. i dont know

From what I saw, yes, that would be the "issue", but it should not be the issue for rendering.

@iss you sure it would not affect rendering? From what I can tell, there's a seek to keyframe (and decode of all following frames up to the one we need) in all cases except if the frame that is needed is exactly "current frame + 1" (anim_movie.cc, ffmpeg_must_seek function). This probably means that if you have a retimed video that is sped up (not slowed down!) by more than 2x, rendering it out will cause every frame to involve a seeking to keyframe operation. If keyframes are placed sparsely, this cost could add up.

> From what I saw, yes, that would be the "issue", but it should not be the issue for rendering. @iss you sure it would not affect rendering? From what I can tell, there's a seek to keyframe (and decode of all following frames up to the one we need) in all cases except if the frame that is needed is exactly "current frame + 1" (`anim_movie.cc`, `ffmpeg_must_seek` function). This *probably* means that if you have a retimed video that is *sped up* (not slowed down!) by more than 2x, rendering it out will cause every frame to involve a seeking to keyframe operation. If keyframes are placed sparsely, this cost could add up.

@aras_p Sorry, I meant "yes it did affect the rendering, but it should not do that". Now it should be fixed at least.
The seeking code is optimized for doing larger steps. Seeking itself is not costly operation. But decoder must read frames in order from keyframe to requested timestamp, otherwise you get glitches.
Current code checks, if after seeking the "current keyframe" was changed. This is facilitated by ffmpeg_seek_buffers_need_flushing(). If this wasn't working correctly, you would notice this also when scrubbing the timeline - moving playhead forward would be as slow as moving it backwards.

Just to clarify, the issue fixed by 1e6f21c34d was, that currently ffmpeg did not have option to not run ffmpeg_decode_video_frame_scan() which requires advancing the frame at least by 1. So now you can request the same frame over and over, and it will just skip decoding alltogether. This is possibly useful for other areas, but not sure if there is place where image is not cached.

@aras_p Sorry, I meant "yes it did affect the rendering, but it should not do that". Now it should be fixed at least. The seeking code is optimized for doing larger steps. Seeking itself is not costly operation. But decoder must read frames in order from keyframe to requested timestamp, otherwise you get glitches. Current code checks, if after seeking the "current keyframe" was changed. This is facilitated by `ffmpeg_seek_buffers_need_flushing()`. If this wasn't working correctly, you would notice this also when scrubbing the timeline - moving playhead forward would be as slow as moving it backwards. Just to clarify, the issue fixed by 1e6f21c34d1681f9d03f27b304e2dfdb3277de92 was, that currently ffmpeg did not have option to not run `ffmpeg_decode_video_frame_scan()` which requires advancing the frame at least by 1. So now you can request the same frame over and over, and it will just skip decoding alltogether. This is possibly useful for other areas, but not sure if there is place where image is not cached.

just want to confirm that after the fix, the render speed of retimed strips improved a lot.
it is still not as fast as most open source video editors out there though. is it because its displaying stuff in image editor? 🤔

anyway, thx guys. 👍

just want to confirm that after the fix, the render speed of retimed strips improved a lot. it is still not as fast as most open source video editors out there though. is it because its displaying stuff in image editor? 🤔 anyway, thx guys. 👍
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
5 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#126472
No description provided.