Realtime Compositor: Support full float compositing #113476

Merged
Omar Emara merged 10 commits from OmarEmaraDev/blender:gpu-full-float-compositor into main 2023-11-08 08:32:10 +01:00
Member

This patch adds support for full float compositing for the Realtime
Compositor. A new precision option was added to the experimental GPU
render compositor to change between half and full float compositing,
while the viewport compositor remains unchanged at half float for now.

The compositor context now need to implement the get_precision() method
to indicate its preferred precision. Intermediate results will be stored
using the context's precision, with a number of exceptions that can use
a different precision regardless of the context's precision. For
instance, summed area tables are always stored in full float results
even if the context specified half float. Conversely, jump flooding
tables are always stored in half integer results even if the context
specified full float. The former requires full float while the latter
has no use for it.

Since shaders are created for a specific precision, we need two variants
of each compositor shader to account for the context's possible
precision. However, to avoid doubling the shader info count and reduce
boilerplate code and development time, an automated mechanism was
employed. A single shader info of whatever precision needs to be added,
then, at runtime, the shader info can be adjusted to change the
precision of the outputs. That shader variant is then cached in the
static cache manager for future processing-free shader retrieval.
Therefore, the shader manager was removed in favor of a cached shader
container in the static cache manager.

A number of utilities were added to make the creation of results as well as
the retrieval of shader with the target precision easier. Further, a
number of precision-specific shaders were removed in favor of more
generic ones that utilizes the aforementioned shader retrieval
mechanism.

This patch adds support for full float compositing for the Realtime Compositor. A new precision option was added to the experimental GPU render compositor to change between half and full float compositing, while the viewport compositor remains unchanged at half float for now. The compositor context now need to implement the get_precision() method to indicate its preferred precision. Intermediate results will be stored using the context's precision, with a number of exceptions that can use a different precision regardless of the context's precision. For instance, summed area tables are always stored in full float results even if the context specified half float. Conversely, jump flooding tables are always stored in half integer results even if the context specified full float. The former requires full float while the latter has no use for it. Since shaders are created for a specific precision, we need two variants of each compositor shader to account for the context's possible precision. However, to avoid doubling the shader info count and reduce boilerplate code and development time, an automated mechanism was employed. A single shader info of whatever precision needs to be added, then, at runtime, the shader info can be adjusted to change the precision of the outputs. That shader variant is then cached in the static cache manager for future processing-free shader retrieval. Therefore, the shader manager was removed in favor of a cached shader container in the static cache manager. A number of utilities were added to make the creation of results as well as the retrieval of shader with the target precision easier. Further, a number of precision-specific shaders were removed in favor of more generic ones that utilizes the aforementioned shader retrieval mechanism.
Omar Emara added the
Interest
Compositing
Module
VFX & Video
labels 2023-10-10 08:10:38 +02:00
Omar Emara added 1 commit 2023-10-10 08:10:49 +02:00
Omar Emara added 2 commits 2023-10-10 16:54:50 +02:00
Omar Emara added 1 commit 2023-10-11 09:46:44 +02:00
Omar Emara added 2 commits 2023-10-11 11:35:42 +02:00
Fix incompatible precision in multiple algorithms
All checks were successful
buildbot/vexp-code-patch-coordinator Build done.
6976f6c2c5
Omar Emara changed title from WIP: Realtime Compositor: Support full float compositing to Realtime Compositor: Support full float compositing 2023-10-11 12:02:02 +02:00
Omar Emara requested review from Sergey Sharybin 2023-10-11 12:22:36 +02:00
Author
Member

@blender-bot package

@blender-bot package
Member

Package build started. Download here when ready.

Package build started. [Download here](https://builder.blender.org/download/patch/PR113476) when ready.

On a code side the change looks good. What I am not sure about is that it is a user selectable option. Is it for the development only or you have some use-case in mind when artists really need to worry about such configuration?

On a code side the change looks good. What I am not sure about is that it is a user selectable option. Is it for the development only or you have some use-case in mind when artists really need to worry about such configuration?
Author
Member

@Sergey I seemed to remember @brecht wanting us to add this option, but maybe I misunderstood.

Regardless, half floats are faster and more memory efficient, so I suspect users might want this option as a sort of optimization when they don't care much about precision.

@Sergey I seemed to remember @brecht wanting us to add this option, but maybe I misunderstood. Regardless, half floats are faster and more memory efficient, so I suspect users might want this option as a sort of optimization when they don't care much about precision.

Artists want speed and precision :)

If we decide on such option maybe we should have "Auto" option? Use half-float in interactive editing (viewport, viewer node) for performance, and full-float during F12 render.

Artists want speed and precision :) If we decide on such option maybe we should have "Auto" option? Use half-float in interactive editing (viewport, viewer node) for performance, and full-float during F12 render.
Author
Member

That might be a good idea. :)

@brecht You said on #99210:

Full float instead of half float precision for final renders and passes that need it.

Did you mean always use full float or have it as an option with maybe an Auto as suggested by Sergey?

That might be a good idea. :) @brecht You said on #99210: > Full float instead of half float precision for final renders and passes that need it. Did you mean always use full float or have it as an option with maybe an Auto as suggested by Sergey?

What I imagined is that we would automatically determine which buffers need float precision. For example ID Mask values should be full float for the viewport too, because the node just doesn't work correctly without that. For this we could propagate backwards from node inputs (e.g. Z in Z Combine, ID in ID Mask), or propagate forwards from certain passes (Z Depth, Position, ID), or both.

So for me the "Auto" was not about distinction between viewport and final compositing. And then you would have an option to switch to full float precision for when those heuristics fail. And I guess render output to file should always be full float precision to preserve as much information as possible.

What I imagined is that we would automatically determine which buffers need float precision. For example ID Mask values should be full float for the viewport too, because the node just doesn't work correctly without that. For this we could propagate backwards from node inputs (e.g. Z in Z Combine, ID in ID Mask), or propagate forwards from certain passes (Z Depth, Position, ID), or both. So for me the "Auto" was not about distinction between viewport and final compositing. And then you would have an option to switch to full float precision for when those heuristics fail. And I guess render output to file should always be full float precision to preserve as much information as possible.
Author
Member

@brecht Those per-socket or per-node precision controls will still be done, but in a separate patch, this patch just lays the groundwork for that. We even already have nodes in that patch that either force half or full float as needed regardless of what the user chooses.

@brecht Those per-socket or per-node precision controls will still be done, but in a separate patch, this patch just lays the groundwork for that. We even already have nodes in that patch that either force half or full float as needed regardless of what the user chooses.

Ok,, I agree with an "Auto" opton that works like this then:

Use half-float in interactive editing (viewport, viewer node) for performance, and full-float during F12 render.

Ok,, I agree with an "Auto" opton that works like this then: > Use half-float in interactive editing (viewport, viewer node) for performance, and full-float during F12 render.

So then it sounds we need only two options Auto and Float? The reasoning being that passes like Cryptomatte have to be float, and forcing those being half is going to lead to issues, but for cases when automatic deduction of type failed give an option to force full precision.

So then it sounds we need only two options `Auto` and `Float`? The reasoning being that passes like Cryptomatte have to be float, and forcing those being half is going to lead to issues, but for cases when automatic deduction of type failed give an option to force full precision.
Author
Member

I have to disagree here. Your reasoning seems backward. The option does not force precision, the sockets and nodes are the ones who can and should force a precision as needed.

For example, if the option is set to Half, passes like Cryptomatte will force full precision and disregard the option set by the user. Conversely, if the option is set to Full, jump flooding will force half precision and disregard the option set by the user.

So the Half option will not cause any issues, while being practically useful. So I don't see why it should be removed.

I have to disagree here. Your reasoning seems backward. The option does not force precision, the sockets and nodes are the ones who can and should force a precision as needed. For example, if the option is set to Half, passes like Cryptomatte will force full precision and disregard the option set by the user. Conversely, if the option is set to Full, jump flooding will force half precision and disregard the option set by the user. So the Half option will not cause any issues, while being practically useful. So I don't see why it should be removed.

I am not sure what the practical use of Half option would be.

The final renders should contain as much information as possible, and this is what the Auto will toggle to. If something in detection fails, there is a way to force precision to be Float.

Likewise, if someone deals with something very high dynamic in viewport (which is defaulted to Half as much as possible) they can force full-float support in viewport for those cases.

But why would one try to force final render to use Half? Or, why would one will ant to toggle the precision option to Half?

I am not sure what the practical use of Half option would be. The final renders should contain as much information as possible, and this is what the Auto will toggle to. If something in detection fails, there is a way to force precision to be Float. Likewise, if someone deals with something very high dynamic in viewport (which is defaulted to Half as much as possible) they can force full-float support in viewport for those cases. But why would one try to force final render to use Half? Or, why would one will ant to toggle the precision option to Half?
Author
Member

Half floats are faster and consume less memory. If a user sets up a pipeline to process 8-bit images, which are then also saved in 8-bit images, where full precision is not really useful, why shouldn't we allow that user to to have a faster and a more memory efficient pipeline?

Further, the fact that Auto can use half float means the result of the compositor previews can be slightly different from final renders in some cases, and if the user likes the half float result better, if we don't have an option for half, that would leave the user unsatisfied. I know we might be tempted to say this doesn't make sense, but it does happen that users might like less accurate results for whatever reason.

Half floats are faster and consume less memory. If a user sets up a pipeline to process 8-bit images, which are then also saved in 8-bit images, where full precision is not really useful, why shouldn't we allow that user to to have a faster and a more memory efficient pipeline? Further, the fact that Auto can use half float means the result of the compositor previews can be slightly different from final renders in some cases, and if the user likes the half float result better, if we don't have an option for half, that would leave the user unsatisfied. I know we might be tempted to say this doesn't make sense, but it does happen that users might like less accurate results for whatever reason.
Omar Emara added 3 commits 2023-11-03 16:11:13 +01:00

I am not sure how common the use-case of handling 8bit images in compositing software is. The amount of such cases, and the performance benefit on one hand, and on another hand a very easy way to loose precision for EXR and render passes inputs doesn't make having such option compelling to me.

The half-float is also something that we currently only do on GPU. So having such option for final render contributes to confusing cases when render result looks different between different setups, or renders from a farm looking different from what artists have.

Such option is not something I'd personally provide. But also don't feel strong enough to keep the discussion going and stalling the rest of the change.

One thing I am confused with the latest update is that it seems to remove check for the use_realtime and experimental flag when exposing the option. Was that an intended change?

I am not sure how common the use-case of handling 8bit images in compositing software is. The amount of such cases, and the performance benefit on one hand, and on another hand a very easy way to loose precision for EXR and render passes inputs doesn't make having such option compelling to me. The half-float is also something that we currently only do on GPU. So having such option for final render contributes to confusing cases when render result looks different between different setups, or renders from a farm looking different from what artists have. Such option is not something I'd personally provide. But also don't feel strong enough to keep the discussion going and stalling the rest of the change. One thing I am confused with the latest update is that it seems to remove check for the `use_realtime ` and experimental flag when exposing the option. Was that an intended change?
Omar Emara added 1 commit 2023-11-03 21:53:17 +01:00
Author
Member

One thing I am confused with the latest update is that it seems to remove check for the use_realtime and experimental flag when exposing the option. Was that an intended change?

The precision now also controls the viewport compositor, which not experimental and is not related to the GPU render compositor.

> One thing I am confused with the latest update is that it seems to remove check for the use_realtime and experimental flag when exposing the option. Was that an intended change? The precision now also controls the viewport compositor, which not experimental and is not related to the GPU render compositor.
Sergey Sharybin approved these changes 2023-11-07 17:38:16 +01:00
Sergey Sharybin left a comment
Owner

The precision now also controls the viewport compositor, which not experimental and is not related to the GPU render compositor.

Ah, lovely!

As for the change, to me it sounds all good. At least on a higher-level review. I didn't do preciese line-by-line hunt for possible bugs, and also did not have time to do user level testing after the recent changes.

Unless you specifically want me to test it on a user level lets go ahead and land it.

> The precision now also controls the viewport compositor, which not experimental and is not related to the GPU render compositor. Ah, lovely! As for the change, to me it sounds all good. At least on a higher-level review. I didn't do preciese line-by-line hunt for possible bugs, and also did not have time to do user level testing after the recent changes. Unless you specifically want me to test it on a user level lets go ahead and land it.
Omar Emara merged commit 474b6fa070 into main 2023-11-08 08:32:10 +01:00
Omar Emara deleted branch gpu-full-float-compositor 2023-11-08 08:32:12 +01:00
Sign in to join this conversation.
No reviewers
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset System
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Code Documentation
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Viewport & EEVEE
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Asset Browser Project
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Asset System
Module
Core
Module
Development Management
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Module
Viewport & EEVEE
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Severity
High
Severity
Low
Severity
Normal
Severity
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#113476
No description provided.