Design: DrawEngine Color Management #72420

New Issue

Jeroen Bakker · 2019-12-13T13:03:22+01:00

Jeroen Bakker commented

2019-12-13 13:03:22 +01:00

After the overlay engine refactoring landed we are one step closer to do better color management in the 3d viewport. This design task describes the steps that needs to happen and the reasoning behind it.

Current Limitation

Colors in the viewport and after rendering are different when using overlay engine. The reason is that color transfer is done as one of the last steps of the first render engine (EEVEE, Workbench). The overlay engine ignores the color space it is compositing into.
Colors in the viewport and after rendering are different when using grease pencil. The reason is that color transfer is done as one of the last steps of the first render engine (EEVEE, Workbench). The render engine can set the output buffer already to sRGB or linear REC709. Grease Pencil (the second render engine) converts its input color to match this behavior, but does the blending in the color space of the buffer.

DrawManager

One benefit of the overlay unification project is that the overlays are rendered in a separate GPU texture. . This was a first step of implementing a better drawing color management pipeline.

This proposal consist of several some rules.

The Draw manager will be responsible for the view transform, pending on the active draw mode and draw engines.
The Draw Manager will be responsible to perform the display transform as a last step in the drawing pipeline.
The compositing between engines must be performed in scene reference space.

Image empties and camera background images are currently composited directly in the engine output. That is a hack and has to be removed.

Conceptually the pipeline would look like:

Note: that the Z Combine could also be any compositing trick. It is controlled by the draw engine. The Z Combine shows when it is being used in the pipeline, not what is being done.
Note: Due to upcoming implementation the view transform will happen after the grease pencil engine

WARNING! as this seems conceptually a good idea, OCIO is limiting this. The best we can do is to apply color management after the grease pencil. But the line renderings will always be drawn in display space. The other option is that the line renderings will also be color managed with the settings that the scene and draw mode requests.

Next Steps

Get input from GPU Viewport, Render & Cycles and Grease Pencil.

After the overlay engine refactoring landed we are one step closer to do better color management in the 3d viewport. This design task describes the steps that needs to happen and the reasoning behind it. **Current Limitation** * Colors in the viewport and after rendering are different when using overlay engine. The reason is that color transfer is done as one of the last steps of the first render engine (EEVEE, Workbench). The overlay engine ignores the color space it is compositing into. * Colors in the viewport and after rendering are different when using grease pencil. The reason is that color transfer is done as one of the last steps of the first render engine (EEVEE, Workbench). The render engine can set the output buffer already to sRGB or linear REC709. Grease Pencil (the second render engine) converts its input color to match this behavior, but does the blending in the color space of the buffer. **DrawManager** One benefit of the overlay unification project is that the overlays are rendered in a separate GPU texture. ![20181213_engine.png](https://archive.blender.org/developer/F8216336/20181213_engine.png) ![20181213_overlay.png](https://archive.blender.org/developer/F8216337/20181213_overlay.png) ![20181213_combined.png](https://archive.blender.org/developer/F8216339/20181213_combined.png). This was a first step of implementing a better drawing color management pipeline. This proposal consist of several some rules. 1. The Draw manager will be responsible for the view transform, pending on the active draw mode and draw engines. 2. The Draw Manager will be responsible to perform the display transform as a last step in the drawing pipeline. 3. The compositing between engines must be performed in scene reference space. - Image empties and camera background images are currently composited directly in the engine output. That is a hack and has to be removed. Conceptually the pipeline would look like: ![image.png](https://archive.blender.org/developer/F8216489/image.png) Note: that the Z Combine could also be any compositing trick. It is controlled by the draw engine. The Z Combine shows when it is being used in the pipeline, not what is being done. Note: Due to upcoming implementation the view transform will happen after the grease pencil engine WARNING! as this seems conceptually a good idea, OCIO is limiting this. The best we can do is to apply color management after the grease pencil. But the line renderings will always be drawn in display space. The other option is that the line renderings will also be color managed with the settings that the scene and draw mode requests. **Next Steps** 1. Get input from `GPU Viewport`, `Render & Cycles` and `Grease Pencil`.

Jeroen Bakker self-assigned this 2019-12-13 13:03:22 +01:00

Jeroen Bakker commented

2019-12-13 13:03:22 +01:00

Added subscriber: @Jeroen-Bakker

Jeroen Bakker changed title from ~~[WIP] Design: DrawEngine Colormanagement~~ to [WIP] Design: DrawEngine Color Management

2019-12-13 13:57:31 +01:00

Jeroen Bakker changed title from ~~[WIP] Design: DrawEngine Color Management~~ to Design: DrawEngine Color Management

2019-12-13 14:25:03 +01:00

item412 commented

2019-12-13 15:06:23 +01:00

Added subscriber: @item412

Pipeliner commented

2019-12-13 17:14:59 +01:00

Added subscriber: @Pipeliner

Evan Wilson commented

2019-12-13 21:51:22 +01:00

Added subscriber: @EAW

Sam Van Hulle commented

2019-12-14 11:41:35 +01:00

Added subscriber: @frameshift

Sam Van Hulle commented

2019-12-15 22:31:28 +01:00

Great to see this being tackled! One small note:

The compositing between engines must be performed in linear REC709.

I believe you mean the scene linear reference space.

Great to see this being tackled! One small note: > The compositing between engines must be performed in linear REC709. I believe you mean *the scene linear reference space*.

Antonio Vazquez commented

2019-12-16 11:52:47 +01:00

Added subscribers: @fclem, @antoniov

Antonio Vazquez commented

2019-12-16 11:52:47 +01:00

@Jeroen-Bakker Actually @fclem is doing a big refactor of the grease pencil drawing engine and he is already thinking in color management.

Clément Foucault commented

2019-12-19 14:12:33 +01:00

Side note: I recently found that the sRGB texture format was allowed to be used in framebuffers (but was listed under texture only fomat in GPU module). This texture format if use used correctly should give us linear blending an decoding of srgb textures for free. (note: i do know we already use that for image textures).

Clément Foucault commented

2020-01-15 00:02:50 +01:00

Ok I've managed to do a proof of concept implementation of the sRGB framebuffer handling and it gives a lot of benefits... but changes some things that we may want to take into account.

Here is the paste with the diff: P1213

New (linear color space blending)*

Old (display space color space blending)

*Note: This is technically true only if the display space is sRGB. If not, opengl will still assume it is and do the sRGB - Linear conversions instead of Display Space - Linear.

We can see that the blending of bright color is more noticeable (i.e: the grid) and the darker ones are less prominent than before (i.e.:the lights' ground line. barely noticeable below selected orange lights).
This gives much more pleasant anti-aliasing which was my motivation for this.

The technical bits:

We need sRGB texture for viewport and overlay render targets.
We need to convert all colors used in overlays to linear colors. Most of them are in draw_common.c but some are still hardcoded (i.e: bone drawing IIRC)
We need to ENABLE GL_FRAMEBUFFER_SRGB when we render to these framebuffers. This way opengl does the Linear to sRGB conversion AFTER blending when doing the writes.
We need to DISABLE GL_FRAMEBUFFER_SRGB inside DRW_transform_to_display because the display transform is already done by the OCIO shader.
We have to convert to sRGB when drawing the viewport area texture since the texture is now sampled and converted to linear space. In my patch I just enabled GL_FRAMEBUFFER_SRGB for this step. As it turns out the default framebuffer render target is sRGB for some reason (this might be implementation dependent so I would not rely on this to be always true).

I tried to always enable GL_FRAMEBUFFER_SRGB (which is supposed to only modify writes to sRGB textures/rendertargets) but now everything in blender's UI need to be in linear space since it is transformed back to sRGB by opengl's fixed pipeline.
You can such missing color correction on the view gizmos in the top right corner of the viewport.

So we can decide to only do the viewport management .

BTW this also mean doing proper color management for the Image editor since it also uses GPUViewport.

I think this is a nice step into a better color managed UI. We could use sRGB for area framebuffer everywhere to hold data in a compressed manner (in GL_SRGB8_A8 format) and use OCIO to do the Linear > display space conversion at blitting time. This way we can make an exception for the viewport by using a floating point format to easily support HDR displays and wide gammuts.

We could also use GL_RGB10_A2format for areas buffers but this means having less alpha precision.

Ok I've managed to do a proof of concept implementation of the sRGB framebuffer handling and it gives a lot of benefits... but changes some things that we may want to take into account. Here is the paste with the diff: [P1213](https://archive.blender.org/developer/P1213.txt) New (linear color space blending)* ![Capture d’écran du 2020-01-14 22-46-44.png](https://archive.blender.org/developer/F8278467/Capture_d_écran_du_2020-01-14_22-46-44.png) Old (display space color space blending) ![Capture d’écran du 2020-01-14 22-47-09.png](https://archive.blender.org/developer/F8278465/Capture_d_écran_du_2020-01-14_22-47-09.png) *Note: This is technically true only if the display space is sRGB. If not, opengl will still assume it is and do the sRGB - Linear conversions instead of Display Space - Linear. We can see that the blending of bright color is more noticeable (i.e: the grid) and the darker ones are less prominent than before (i.e.:the lights' ground line. barely noticeable below selected orange lights). This gives much more pleasant anti-aliasing which was my motivation for this. The technical bits: - We need sRGB texture for viewport and overlay render targets. - We need to convert all colors used in overlays to linear colors. Most of them are in `draw_common.c` but some are still hardcoded (i.e: bone drawing IIRC) - We need to ENABLE `GL_FRAMEBUFFER_SRGB` when we render to these framebuffers. This way opengl does the Linear to sRGB conversion AFTER blending when doing the writes. - We need to DISABLE `GL_FRAMEBUFFER_SRGB` inside `DRW_transform_to_display` because the display transform is already done by the OCIO shader. - We have to convert to sRGB when drawing the viewport area texture since the texture is now sampled and converted to linear space. In my patch I just enabled GL_FRAMEBUFFER_SRGB for this step. As it turns out the default framebuffer render target is sRGB for some reason (this might be implementation dependent so I would not rely on this to be always true). I tried to always enable GL_FRAMEBUFFER_SRGB (which is supposed to only modify writes to sRGB textures/rendertargets) but now everything in blender's UI need to be in linear space since it is transformed back to sRGB by opengl's fixed pipeline. You can such missing color correction on the view gizmos in the top right corner of the viewport. So we can decide to only do the viewport management . BTW this also mean doing proper color management for the Image editor since it also uses GPUViewport. I think this is a nice step into a better color managed UI. We could use sRGB for area framebuffer everywhere to hold data in a compressed manner (in `GL_SRGB8_A8` format) and use OCIO to do the Linear > display space conversion at blitting time. This way we can make an exception for the viewport by using a floating point format to easily support HDR displays and wide gammuts. We could also use `GL_RGB10_A2`format for areas buffers but this means having less alpha precision.

Clément Foucault commented

2020-01-15 20:03:04 +01:00

Added subscriber: @troy_s

Clément Foucault commented

2020-01-15 20:03:04 +01:00

After talking with @troy_s I've come to realize that the terminology is a bit messed up here.

When I was referring to sRGB textures and framebuffers, I was just referring to the transfer function that does not alter the primaries and just remap the input for better perceptual storage. The input color would still be linear.

Also when we say we will render overlays in display space, we mean Linear Display Space.

After talking with @troy_s I've come to realize that the terminology is a bit messed up here. When I was referring to sRGB textures and framebuffers, I was just referring to the transfer function that does not alter the primaries and just remap the input for better perceptual storage. The input color would still be linear. Also when we say we will render overlays in display space, we mean Linear Display Space.

Evan Wilson commented

2020-01-15 21:19:24 +01:00

It is difficult to read white text over a light grey alpha background like Firefox default

so I remade it over a solid black background in case anyone else was having the same difficulty.

It is difficult to read white text over a light grey alpha background like Firefox default ![image.png](https://archive.blender.org/developer/F8216489/image.png) so I remade it over a solid black background in case anyone else was having the same difficulty. ![Pipeline_as_Nodes_over_Black.png](https://archive.blender.org/developer/F8280604/Pipeline_as_Nodes_over_Black.png)

Troy Sobotka commented

2020-01-15 22:01:09 +01:00

I've included the relevant colourimetric transforms required in the chain, given three two different layers of transparency in the viewport. Hope someone finds this informative.

I've included the relevant colourimetric transforms required in the chain, given three two different layers of transparency in the viewport. Hope someone finds this informative. ![Blender Pixel Merge Overview.png](https://archive.blender.org/developer/F8280670/Blender_Pixel_Merge_Overview.png)

Benjamin Sannholm commented

2020-06-18 19:15:05 +02:00

Added subscriber: @BSannholm

Jeroen Bakker commented

2020-06-25 08:28:48 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Jeroen Bakker closed this issue

2020-06-25 08:28:48 +02:00

Pauan commented

2021-09-29 16:17:52 +02:00

Added subscriber: @pauanyu_blender

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Design: DrawEngine Color Management #72420