VSE: confusing strip color manipulation #110635

New Issue

Hannes Loeschke · 2023-07-30T22:04:50+02:00

Hannes Loeschke commented

2023-07-30 22:04:50 +02:00

System Information
Operating system: Windows 11
Graphics card: RTX 4060 Laptop GPU

Blender Version
Broken: all current versions
Worked: couldn't find any

Short description of error
I am using D-Log DJI drone footage. Finding and applying the correct color transform is a pain in the a.. but I got it working.
My footage is a bit overexposed so I tried the obvious: "Color > Multiply" to reduce EV.
For values below 1.0 the Alpha channel ist reduced.
I could not find anything on the intent of this issue. To me this makes no sense:

most video formats don't have an alpha channel
overall alpha of the strip is adjusted in the Blend Mode options
saturation doesn't affect the alpha channel, so why should applying a factor in the same tab named "Color"?

Exact steps for others to reproduce the error

load any HDR footage, that will need some exposure reduction
In the strip properties set Color > Multiply to a value below 1.0

Side note
Saturation is applied before "conversion to float" if checked.
My footage is already in float. So color space transform obviously takes place before entering preprocessing stage. Looking at the code, the conversion to float tries to transform to sequencer color space if applicaple. Depending on the color space configuartion, the transform can happen before or after modifying the saturation.
Also: some combinations of strip and sequencer color space yield different results for "color>multiply" with and without "convert to float". But I need to test this some more.

**System Information** Operating system: Windows 11 Graphics card: RTX 4060 Laptop GPU **Blender Version** Broken: all current versions Worked: couldn't find any **Short description of error** I am using D-Log DJI drone footage. Finding and applying the correct color transform is a pain in the a.. but I got it working. My footage is a bit overexposed so I tried the obvious: "Color > Multiply" to reduce EV. For values below 1.0 the Alpha channel ist reduced. I could not find anything on the intent of this issue. To me this makes no sense: - most video formats don't have an alpha channel - overall alpha of the strip is adjusted in the Blend Mode options - saturation doesn't affect the alpha channel, so why should applying a factor in the same tab named "Color"? **Exact steps for others to reproduce the error** - load any HDR footage, that will need some exposure reduction - In the strip properties set Color > Multiply to a value below 1.0 **Side note** Saturation is applied before "conversion to float" if checked. My footage is already in float. So color space transform obviously takes place before entering preprocessing stage. Looking at the code, the conversion to float tries to transform to sequencer color space if applicaple. Depending on the color space configuartion, the transform can happen before or after modifying the saturation. Also: some combinations of strip and sequencer color space yield different results for "color>multiply" with and without "convert to float". But I need to test this some more.

Hannes Loeschke added the

labels 2023-07-30 22:04:51 +02:00

YimingWu commented

2023-07-31 05:12:25 +02:00

Can confirm that this applies to all types of strip, not limited to image or HDR videos. IMO multiplying in the color section should not affect alpha.

Looks like it uses a simple approach of multiplying all imbuf channels in source/blender/sequencer/intern/render.cc:

  float mul = seq->mul;
  if (seq->blend_mode == SEQ_BLEND_REPLACE) {
    mul *= seq->blend_opacity / 100.0f;
  }

  if (mul != 1.0f) {
    multibuf(preprocessed_ibuf, mul); // <---- it multiplies all 4 channels
  }

I think the best way to do this is just use the mul value on RGB channel and use the opacity on A channel, but this can break existing files without versioning. (Or is there any pre-multiply magic going on here?)

@iss any ideas?

Can confirm that this applies to all types of strip, not limited to image or HDR videos. IMO multiplying in the color section should not affect alpha. Looks like it uses a simple approach of multiplying all `imbuf` channels in `source/blender/sequencer/intern/render.cc`: ``` float mul = seq->mul; if (seq->blend_mode == SEQ_BLEND_REPLACE) { mul *= seq->blend_opacity / 100.0f; } if (mul != 1.0f) { multibuf(preprocessed_ibuf, mul); // <---- it multiplies all 4 channels } ``` I think the best way to do this is just use the mul value on `RGB` channel and use the opacity on `A` channel, but this can break existing files without versioning. (Or is there any pre-multiply magic going on here?) @iss any ideas?

YimingWu added

and removed

labels 2023-07-31 05:12:39 +02:00

Hannes Loeschke commented

2023-07-31 09:58:16 +02:00

Looks like it uses a simple approach of multiplying all imbuf channels in source/blender/sequencer/intern/render.cc:

Yes. I tried to track down any reason for that. I cannot find a reasoning for applying the factor to alpha and documantation doesn't even mention alpha and only refers to "increasing the brightness".

  float mul = seq->mul;
  if (seq->blend_mode == SEQ_BLEND_REPLACE) {
    mul *= seq->blend_opacity / 100.0f;
  }

On further thought, this looks like another problem. The opacity of BLEND_REPLACE is baked into the image by fading it to black. This is okay if you ignore the alpha channel. Ifalpha channel is preserved in the output, the blend_opacity is applied twice. Once by color multiplication and once in the alpha channel.

  if (mul != 1.0f) {
    multibuf(preprocessed_ibuf, mul); // <---- it multiplies all 4 channels
  }
I think the best way to do this is just use the mul value on RGB channel and use the opacity on A channel, but this can break existing files without versioning. (Or is there any pre-multiply magic going on here?)

For any factor larger than 1.0 nothing would change since alpha is generally clamped to 1.0. Excecpt for fringe cases where the strip actually has an alpha mask. In this case, any feathering in the mask would no longer blow out,.
The question is if anyone actually used color multiply with a factor lower than 1 in a meaningful way. The color multiply is not really documented for values below 1.0 and there are no tutorials using it (that I could find) so the impact is probaly close to zero.

> Looks like it uses a simple approach of multiplying all `imbuf` channels in `source/blender/sequencer/intern/render.cc`: Yes. I tried to track down any reason for that. I cannot find a reasoning for applying the factor to alpha and documantation doesn't even mention alpha and only refers to "increasing the brightness". > ``` > float mul = seq->mul; > if (seq->blend_mode == SEQ_BLEND_REPLACE) { > mul *= seq->blend_opacity / 100.0f; > } > ``` On further thought, this looks like another problem. The opacity of BLEND_REPLACE is baked into the image by fading it to black. This is okay if you ignore the alpha channel. Ifalpha channel is preserved in the output, the blend_opacity is applied twice. Once by color multiplication and once in the alpha channel. > ``` > if (mul != 1.0f) { > multibuf(preprocessed_ibuf, mul); // <---- it multiplies all 4 channels > } > ``` > > I think the best way to do this is just use the mul value on `RGB` channel and use the opacity on `A` channel, but this can break existing files without versioning. (Or is there any pre-multiply magic going on here?) For any factor larger than 1.0 nothing would change since alpha is generally clamped to 1.0. Excecpt for fringe cases where the strip actually has an alpha mask. In this case, any feathering in the mask would no longer blow out,. The question is if anyone actually used color multiply with a factor lower than 1 in a meaningful way. The color multiply is not really documented for values below 1.0 and there are no tutorials using it (that I could find) so the impact is probaly close to zero.

👍 1

Richard Antalik commented

2023-07-31 19:58:58 +02:00

Multiplication with and without affecting alpha are used for number of different goals. Adjusting exposure by simple multiplication I think would be OK in linear colorspace, but for sRGB footage I would rather do this as part of colormanagement.

In any case, I don't think we will know how this feature was intended to be used exactly. But I would agree, that multiply perhaps should operate on RGB channels only, similar to compositor. Personally, I don't think, that breaking compability in this case would cause issues in most files.

So I will check OCIO features first for EV, and only modify multiply feature if OCIO does not help in this case. Or perhaps I could re-purpose multiplication for exposure as that was likely primary use-case.

Multiplication with and without affecting alpha are used for number of different goals. Adjusting exposure by simple multiplication I think would be OK in linear colorspace, but for sRGB footage I would rather do this as part of colormanagement. In any case, I don't think we will know how this feature was intended to be used exactly. But I would agree, that multiply perhaps should operate on RGB channels only, similar to compositor. Personally, I don't think, that breaking compability in this case would cause issues in most files. So I will check OCIO features first for EV, and only modify multiply feature if OCIO does not help in this case. Or perhaps I could re-purpose multiplication for exposure as that was likely primary use-case.

Hannes Loeschke commented

2023-07-31 21:24:51 +02:00

Multiplication with and without affecting alpha are used for number of different goals. Adjusting exposure by simple multiplication I think would be OK in linear colorspace, but for sRGB footage I would rather do this as part of colormanagement.

My guess is, multiplication and saturation were some basic color adjustment functionality since before OCIO was adopted. I can't trace it bac far enough to verify though.

In any case, I don't think we will know how this feature was intended to be used exactly. But I would agree, that multiply perhaps should operate on RGB channels only, similar to compositor. Personally, I don't think, that breaking compability in this case would cause issues in most files.

So I will check OCIO features first for EV, and only modify multiply feature if OCIO does not help in this case. Or perhaps I could re-purpose multiplication for exposure as that was likely primary use-case.

I'm new to collaborative coding but have some years of experience coding on my own.
I'd like to tackle this as a small first contribution if you don't mind.

> Multiplication with and without affecting alpha are used for number of different goals. Adjusting exposure by simple multiplication I think would be OK in linear colorspace, but for sRGB footage I would rather do this as part of colormanagement. My guess is, multiplication and saturation were some basic color adjustment functionality since before OCIO was adopted. I can't trace it bac far enough to verify though. > In any case, I don't think we will know how this feature was intended to be used exactly. But I would agree, that multiply perhaps should operate on RGB channels only, similar to compositor. Personally, I don't think, that breaking compability in this case would cause issues in most files. > > So I will check OCIO features first for EV, and only modify multiply feature if OCIO does not help in this case. Or perhaps I could re-purpose multiplication for exposure as that was likely primary use-case. I'm new to collaborative coding but have some years of experience coding on my own. I'd like to tackle this as a small first contribution if you don't mind.

👍 1

Richard Antalik commented

2023-08-02 00:41:46 +02:00

I definitele wouldn't mind contributions in this area. If you need any help, feel free to ask here or on blender.chat platform(same handle).

I would encourage you to follow https://wiki.blender.org/wiki/Building_Blender which is easy, and https://wiki.blender.org/wiki/Process/Contributing_Code. There are some guidelines to follow, but for smaller patches it is mostly common sense stuff.

I definitele wouldn't mind contributions in this area. If you need any help, feel free to ask here or on blender.chat platform(same handle). I would encourage you to follow https://wiki.blender.org/wiki/Building_Blender which is easy, and https://wiki.blender.org/wiki/Process/Contributing_Code. There are some guidelines to follow, but for smaller patches it is mostly common sense stuff.

~~Hannes Loeschke referenced this issue 2023-08-09 22:05:59 +02:00~~

Fix #110635: VSE: strip color multiplication affecting alpha channel #110984

Hannes Loeschke referenced a pull request that will close this issue

2023-08-09 22:11:22 +02:00

Fix #110635: VSE: strip color multiplication affecting alpha channel #110984

Richard Antalik referenced this issue from a commit

2023-08-18 12:36:12 +02:00

Fix #110635: VSE: strip color multiplication affecting alpha channel

Richard Antalik closed this issue

2023-08-18 12:36:13 +02:00

Blender Bot added

Status