ffmpeg: multithreaded conversion from RGBA into encoding format #116008

Aras Pranckevicius · 2023-12-10T19:42:37+01:00

2023-12-10 19:42:37 +01:00

Whenever movie frame encoding needs to be in non-RGBA format (pretty much always, e.g. H.264 uses YUV etc.), the ffmpeg code has been using sws_scale() since 2007. But that one is completely single threaded.

It can be multi-threaded by passing "threads" option to SwsContext (in a cumbersome way), combined with sws_scale_frame API. Which however requires frame data buffers to be allocated via AVBuffer machinery.

Rendering a 300-frame part of Sprite Fright Edit (target H.264 Medium):

Windows Ryzen 5950X: 16.1 -> 12.0 seconds (generate_video_frame part 4.7 -> 0.7 sec).
Mac M1 Max: 13.1 -> 12.5 sec. Speedup is smaller, but comparatively, entirely other part of movie rendering (audio resampling inside audaspace) is way more expensive compared to the windows machine.

Whenever movie frame encoding needs to be in non-RGBA format (pretty much always, e.g. H.264 uses YUV etc.), the ffmpeg code has been using `sws_scale()` since 2007. But that one is completely single threaded. It can be multi-threaded by passing "threads" option to `SwsContext` (in a cumbersome way), combined with `sws_scale_frame` API. Which however requires frame data buffers to be allocated via `AVBuffer` machinery. Rendering a 300-frame part of Sprite Fright Edit (target H.264 Medium): - Windows Ryzen 5950X: 16.1 -> 12.0 seconds (`generate_video_frame` part 4.7 -> 0.7 sec). - Mac M1 Max: 13.1 -> 12.5 sec. Speedup is smaller, but comparatively, entirely other part of movie rendering (audio resampling inside audaspace) is way more expensive compared to the windows machine.

👍 3 👀 2 🚀 3

Aras Pranckevicius added 1 commit 2023-12-10 19:42:47 +01:00

buildbot/vexp-code-patch-lint Build done. Details

buildbot/vexp-code-patch-darwin-x86_64 Build done. Details

buildbot/vexp-code-patch-darwin-arm64 Build done. Details

buildbot/vexp-code-patch-linux-x86_64 Build done. Details

buildbot/vexp-code-patch-coordinator Build done. Details

feddb39d72 ffmpeg: multithreaded conversion from RGBA into encoding format

Whenever movie frame encoding needs to be in non-RGBA format (pretty
much always, e.g. H.264 uses YUV etc.), the ffmpeg code has been
sws_scale() since 2007. But that one is completely single threaded.

It can be multi-threaded by passing "threads" option to SwsContext
(in a cumbersome way), and using `sws_scale_frame` API instead. Which
however requires frame data buffers to be allocated via AVBuffer
machinery.

Rendering a 300-frame part of Sprite Fright Edit (target H.264 Medium),
on Windows Ryzen 5950X: 16.1 -> 12.0 seconds (generate_video_frame
part 4.7 -> 0.7 sec).

Aras Pranckevicius commented

2023-12-10 19:53:39 +01:00

@blender-bot build

Aras Pranckevicius changed title from ~~WIP: ffmpeg: multithreaded conversion from RGBA into encoding format~~ to ffmpeg: multithreaded conversion from RGBA into encoding format

2023-12-11 05:52:28 +01:00

Aras Pranckevicius added this to the Video Sequencer project 2023-12-11 05:52:37 +01:00

Aras Pranckevicius requested review from Richard Antalik 2023-12-11 05:52:45 +01:00

Aras Pranckevicius requested review from Sergey Sharybin 2023-12-11 05:52:54 +01:00

Richard Antalik approved these changes 2023-12-15 08:57:04 +01:00

Richard Antalik left a comment

I did not realize, there is multithreaded version of sws_scale. I would assume, that this may have even greater impact when used for decoding in anim_movie.cc

I did not realize, there is multithreaded version of `sws_scale`. I would assume, that this may have even greater impact when used for decoding in `anim_movie.cc`

Aras Pranckevicius commented

2023-12-15 09:51:32 +01:00