WIP: VSE: speedup Alpha Over blend with SIMD #115892

Aras Pranckevicius · 2023-12-07T13:32:30+01:00

2023-12-07 13:32:30 +01:00

Alpha Over is default blend mode used by all strips (images, text etc.), so while SIMD arguably could be used in all of them, this one is probably most important.

Makes straight_uchar_to_premul_float + actual blending math + premul_float_to_straight_uchar use SIMD in a 4-wide fashion, i.e. still works on one pixel at a time.

At 4K UHD resolution, Windows Ryzen 5950X:

do_alphaover_effect_byte time 6.82ms -> 4.93ms
with four image strips, sequencer playback 15.9FPS -> 17.9FPS (there are other time costs, not just alpha over blending)

Same setup on a Mac M1 Max: sequencer playback 15.2FPS -> 18.1FPS

Code design question

The added functions straight_uchar_to_premul_float_simd and premul_float_to_straight_uchar_simd would be generally useful in other places (90% within Sequencer code only though). But it's a good question where to put them:

Putting them into BLI_math_color.h has large downside if needing to include BLI_simd.h which drags in <emmintrin.h> or <sse2neon.h> depending on the platform, both of which are very large headers.
I could put them into some other new header, e.g. hypothetically BLI_math_color_simd.h or similar.
Or should it just be kept only within Sequencer code only? (as is in the PR right now)

Alpha Over is default blend mode used by all strips (images, text etc.), so while SIMD arguably could be used in all of them, this one is probably most important. Makes `straight_uchar_to_premul_float` + actual blending math + `premul_float_to_straight_uchar` use SIMD in a 4-wide fashion, i.e. still works on one pixel at a time. At 4K UHD resolution, Windows Ryzen 5950X: - `do_alphaover_effect_byte` time 6.82ms -> 4.93ms - with four image strips, sequencer playback 15.9FPS -> 17.9FPS (there are other time costs, not just alpha over blending) Same setup on a Mac M1 Max: sequencer playback 15.2FPS -> 18.1FPS **Code design question** The added functions `straight_uchar_to_premul_float_simd` and `premul_float_to_straight_uchar_simd` would be generally useful in other places (90% within Sequencer code only though). But it's a good question where to put them: - Putting them into `BLI_math_color.h` has large downside if needing to include `BLI_simd.h` which drags in `<emmintrin.h>` or `<sse2neon.h>` depending on the platform, both of which are very large headers. - I could put them into some other new header, e.g. hypothetically `BLI_math_color_simd.h` or similar. - Or should it just be kept only within Sequencer code only? (as is in the PR right now)

❤️ 2

Aras Pranckevicius added 1 commit 2023-12-07 13:32:40 +01:00

buildbot/vexp-code-patch-linux-x86_64 Build done. Details

buildbot/vexp-code-patch-darwin-x86_64 Build done. Details

buildbot/vexp-code-patch-windows-amd64 Build done. Details

buildbot/vexp-code-patch-darwin-arm64 Build done. Details

buildbot/vexp-code-patch-coordinator Build done. Details

20b2d709ab VSE: speedup Alpha Over blend with SIMD

Alpha Over is default blend mode used by all strips (images, text etc.),
so while SIMD arguably could be used in all of them, this one is
probably most important.

Makes straight_uchar_to_premul_float + actual blending math +
premul_float_to_straight_uchar use SIMD in a 4-wide fashion, i.e. still
works on one pixel at a time.

At 4K UHD resolution, Windows Ryzen 5950X:
- do_alphaover_effect_byte time 6.82ms -> 4.93ms
- with four image strips, sequencer playback 15.9FPS -> 17.9FPS
  (there are other time costs, not just alpha over blending)

Aras Pranckevicius changed title from ~~VSE: speedup Alpha Over blend with SIMD~~ to WIP: VSE: speedup Alpha Over blend with SIMD

2023-12-07 13:33:12 +01:00

Aras Pranckevicius commented

2023-12-07 13:33:35 +01:00

@blender-bot build

Aras Pranckevicius changed title from ~~WIP: VSE: speedup Alpha Over blend with SIMD~~ to VSE: speedup Alpha Over blend with SIMD

2023-12-07 16:15:30 +01:00

Aras Pranckevicius added this to the Video Sequencer project 2023-12-07 16:15:40 +01:00

Aras Pranckevicius requested review from Richard Antalik 2023-12-07 16:15:54 +01:00

Richard Antalik commented

2023-12-07 17:29:42 +01:00

do_alphaover_effect_byte time 6.82ms -> 4.93ms

That does not sound like much. I was thinking, whether there is chance, that compiler does optimize these to simd already. AFAIK it should especially if code in question is pure for loop, but I think we only call for SSE optimization. Will do some testing locally.

The added functions straight_uchar_to_premul_float_simd and premul_float_to_straight_uchar_simd would be generally useful in other places (90% within Sequencer code only though). But it's a good question where to put them.

BLI_math_color.h sounds like good place to me.

More in general I wanted to suggest making pure simd functions and don't just inline the code if that is practical and possible. This is already done by #if BLI_HAVE_SSE2 in BLI_math_color.h.

> do_alphaover_effect_byte time 6.82ms -> 4.93ms That does not sound like much. I was thinking, whether there is chance, that compiler does optimize these to simd already. AFAIK it should especially if code in question is pure for loop, but I think we only call for SSE optimization. Will do some testing locally. > The added functions straight_uchar_to_premul_float_simd and premul_float_to_straight_uchar_simd would be generally useful in other places (90% within Sequencer code only though). But it's a good question where to put them. `BLI_math_color.h` sounds like good place to me. More in general I wanted to suggest making pure simd functions and don't just inline the code if that is practical and possible. This is already done by `#if BLI_HAVE_SSE2` in `BLI_math_color.h`.

👍 1

Aras Pranckevicius changed title from ~~VSE: speedup Alpha Over blend with SIMD~~ to WIP: VSE: speedup Alpha Over blend with SIMD

2023-12-08 13:26:14 +01:00

Aras Pranckevicius commented

2023-12-08 13:27:05 +01:00

Hold on, while I think the idea makes sense, I'll also try to do the same using OIIO simd.h and see whether that achieves same performance with way more readable code.

Hold on, while I think the idea makes sense, I'll also try to do the same using OIIO `simd.h` and see whether that achieves same performance with way more readable code.

Aras Pranckevicius commented

2023-12-10 13:02:33 +01:00

Hmm so OIIO simd.h is mixed bag. It's fairly extensive, large (perhaps too large for Blender's needs), but at the same time NEON coverage is not their primary interest, plus they are lacking some efficient ways of e.g. converting "packed bytes to a float4" and back, which is what you need in images all the time (they do have those, but e.g. for AVX2 or AVX512 minspec, both of which Blender does not have). I have submitted some PRs upstream there to add some lacking bits, but then we get into a "ok so which exactly OIIO version are we compiling against" mess (some distros build Blender using external OIIO that is just "some other version").

I'm thinking that for Blender's needs, a much smaller simd library would be much better. No versioning conundrums, and it would do exactly what is needed.

But also at the same time, SIMD in VSE effects code does not give huge benefits in most of them, since they are more or less memory bandwidth limited, i.e. amount of math that is being done is pretty small compared to amount of memory traffic. So maybe I'll park this for now.

Hmm so OIIO simd.h is mixed bag. It's fairly extensive, large (perhaps too large for Blender's needs), but at the same time NEON coverage is not their primary interest, plus they are lacking some efficient ways of e.g. converting "packed bytes to a float4" and back, which is what you need in images all the time (they do have those, but e.g. for AVX2 or AVX512 minspec, both of which Blender does not have). I have submitted some PRs upstream there to add some lacking bits, but then we get into a "ok so which exactly OIIO version are we compiling against" mess (some distros build Blender using external OIIO that is just "some other version"). I'm thinking that for Blender's needs, a much smaller simd library would be much better. No versioning conundrums, and it would do exactly what is needed. But also at the same time, SIMD in VSE effects code does not give huge benefits in most of them, since they are more or less memory bandwidth limited, i.e. amount of math that is being done is pretty small compared to amount of memory traffic. So maybe I'll park this for now.

buildbot/vexp-code-patch-linux-x86_64 Build done.

Details

buildbot/vexp-code-patch-darwin-x86_64 Build done.

Details

buildbot/vexp-code-patch-windows-amd64 Build done.

Details

buildbot/vexp-code-patch-darwin-arm64 Build done.

Details

buildbot/vexp-code-patch-coordinator Build done.

Details

This pull request has changes conflicting with the target branch.

source/blender/sequencer/intern/effects.cc

View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u vse-alphaover-opt:aras_p-vse-alphaover-opt

git checkout aras_p-vse-alphaover-opt

Sign in to join this conversation.

No reviewers

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

WIP: VSE: speedup Alpha Over blend with SIMD #115892

Checkout