Path Guiding: Adding guiding on glossy surfaces via RIS #107782
Merged
Sebastian Herholz
merged 1 commits from 2023-05-22 16:47:10 +02:00
sherholz/blender:cycles_path_guiding_RIS
into main
Reviewers
Request review
No reviewers
Labels
Clear labels
This issue affects/is about backward or forward compatibility
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Apply labels
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
This issue affects/is about backward or forward compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Issues relating to security: https://wiki.blender.org/wiki/Process/Vulnerability_Reports
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest: Wayland
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
Milestone
Set milestone
Clear milestone
No items
No Milestone
Projects
Set Project
Clear projects
No project
Assignees
Assign users
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.
No due date set.
Dependencies
No dependencies set.
Reference: blender/blender#107782
Reference in New Issue
There is no content yet.
Delete Branch "sherholz/blender:cycles_path_guiding_RIS"
Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?
This PR adds support for path guiding on glossy materials.
Two new guiding modes are added
Re-sampled Importance Sampling
(RIS) andRoughness-based
.The first uses Re-Sampling Importance Sampling to with two candidates (one from the BSDF and one from the guiding distribution).
the second adjusts the guiding probability based on material roughness (specular -> no guiding and diffuse -> guiding (50%)).
After some intensive tests by the community (https://blenderartists.org/t/cycles-path-guiding-tests/1388137)
I choose to set
RIS
as the new default method.Todo:
surface_shader.h
Example:

UI-changes:

It is a bit hard to see the exact difference when images are put on the same canvas. It would be helpful to have them as individual images to help doing pixel-wise comparison.
For the patch itself I do not find the direction of adding more options something that fits the Blender vision, as well as it adds permutations for developers to ensure be working.
If there is a strong case why all of the guiding methods are needed then it needs to be documented, ideally in a way that is clear for artists.
Otherwise I'd propose to stick to one method and focus on ensuring it works the best possible way.
What I am also curios is the performance comparison on GPU.
@ -21,9 +21,33 @@
CCL_NAMESPACE_BEGIN
#define RIS_COSINE
Is it possible to remove the permutation of the
RIS_COSINE
andRIS_INCOMMING_RADIANCE
?Done,
The other define
# define RIS_INCOMING_RADIANCE
will be gone after we bumped the Open PGL to 0.5.0@ -22,2 +22,4 @@
CCL_NAMESPACE_BEGIN
#define RIS_COSINE
#if OPENPGL_VERSION_MINOR >= 5
I think we can make the 0.5 version required.
Other than that, I am not a fan of such implicit assumption that the
OPENPGL_VERSION_MAJOR
is 0. We really need to have an utility macro like I've shown in the comment in the #106861.I already talked with Ray. As soon as every dependency package updated to Open PGL 0.5 I would like to bump the minimal required version to 0.5 and get rid of all these defines.
OpenPGL was updated to 0.5 for all platforms, so this can be simplified.
@ -530,2 +594,4 @@
return label;
}
ccl_device int surface_shader_bsdf_guided_sample_closure_ris(KernelGlobals kg,
Is there a link to a paper we can add here to help understanding all the math which is going on in the function?
This is an extension/variant of the standard RIS where I added some stuff.
A detailed description of the method with all the insides will be part of the Siggraph2023 Course on path guiding, I can add a link as soon as the course notes are online.
@ -532,0 +639,4 @@
int label_ris[2];
float3 omega_in_ris[2];
float ris_pdfs[2] = {0.f, 0.f};
Here and everywhere else in the patch: do not omit the 0 suffix (0.0f, not 0.f).
This is per the style guide: https://wiki.blender.org/wiki/Style_Guide/C_Cpp#Value_Literals
Done
@Sergey I understand your concerns, ideally I would also prefer to only have one guiding on/off button.
Unfortunately, the is no
THE IDEAL WAY
to integrate path guiding into a production renderer and I try to keep some option, also for debugging and testing purposes.It helped a lot to get feedback from the community in nailing down which version seems to work best in practice.
As you can see all these options are only available under the Debug options, which should not be touched be the artists in their daily work. Unfortunately they do not really listen :( and play around with every button they find :).
I was thinking about hiding these options behind the
WITH_CYCLES_DEBUG
flag so that it is still possible to test and validate different options.For GPU timings we will see when we have GPU support in Open PGL.
I am currently refactoring the RIS code, I think when porting to the GPU it can be made a little bit more GPU friendly when combined with reservoir sampling.
About the examples, here are some links from the community tests:
https://blenderartists.org/t/cycles-path-guiding-tests/1388137/673
https://blenderartists.org/t/cycles-path-guiding-tests/1388137/659
https://blenderartists.org/t/cycles-path-guiding-tests/1388137/656
https://blenderartists.org/t/cycles-path-guiding-tests/1388137/597
Thanks for the updates.
The point about the non-existing ideal algorithm applies to other areas of 3d software as well. It is fine to have extra options during development and feedback period, but for the final merge we really try hard to avoid adding them.
As you've said, no matter how well you hide them, artists use all the options, so there is always a maintenance cost (which is not only limited to code side, but also communication with artists).
The extra annoying part in the Cycles kernel is also that unused code paths could affect performance on GPU, so we always look at those with extra care.
Is it possible to limit options to 2: RIS (the officially used one) and some simple implementation to help nailing down issues between Cycles side, OpenPGL side, and weight computation for the guiding?
Also, to confirm, all the changes are purely within surrounding
__PATH_GUIDING__
ifdef, so no changes for the current GPUs? (is just something that is not so trivially visible from the diff, so wanted to confirm with you).Btw, I've also notice you're using
std::isfinite
. This would not work on all devices/compilers we support as we use-fast-math
. The portable way to do so in Cycles isisnan_safe
andisfinite_safe
.We talked with Sebastian outside of the code review page.
The important point I understand is that we might want to come back to the MIS/Roughness approaches later on.
We also confirmed all the current changes do not affect current GPU code at all.
The
WITH_CYCLES_DEBUG
idea seems interesting, but might be better to handle it separately from this patch.So that pretty much resolves majority of my questions :)
The remaining code aspect I've noted is the finite/nan story. To me it seems easier to just use our "safe" variants, and safe hair-pulling later on.
Sebastian mentioned some cleanup is still planned, so I'll wait for the patch to leave the WIP state to do the final pass of review :)
@blender-bot package
Package build started. Download here when ready.
9a62522681
toaea5dc6ab9
@blender-bot package
Package build started. Download here when ready.
aea5dc6ab9
to2f30bd90b1
@blender-bot package
Package build started. Download here when ready.
WIP: Path Guiding: Adding guiding on glossy surfaces via RISto Path Guiding: Adding guiding on glossy surfaces via RIS@ -14,0 +37,4 @@
Spectrum eval{zero_spectrum()};
};
ccl_device_forceinline bool calculate_ris_target(ccl_private GuidingRISSample *risSample,
We use snake style for variables, so
ris_sample
.Done
@ -67,3 +98,2 @@
/* Init guiding (diffuse BSDFs only for now). */
if (!(diffuse_sampling_fraction > 0.0f &&
guiding_bsdf_init(kg, state, sd->P, sd->N, rand_bsdf_guiding)))
if (!fully_opaque || avg_roughness < guiding_roughness_threshold * guiding_roughness_threshold ||
Units seems a bit odd: comparison of toughness with roughness^2. Is this expected?
This is due to the fact that Blender/Cycles uses
roughness^2
internally and squared roughness returnsalpha.x * alpha.y
so it is actually(roughness^2)^2
compared to the user input.I refactored everything to make this more clear and convert the
guiding_roughness_threshold
to beguiding_roughness_threshold^2
.@ -532,0 +712,4 @@
sum_ris_weights += risSamples[1].ris_weight;
num_ris_candidates++;
}
kernel_assert(sum_ris_weights >= 0.0f);
Should this be instead
kernel_assert(risSamples[1].ris_weight >= 0.0f)
?Otherwise if the first RIS sample weight is 0.5, and the second RIS sample is -0.1 we will not catch the issue with this assert.
I agree, I changed it to test in both cases for the
ris_weight
as well@ -532,0 +806,4 @@
unguided_bsdf_pdf,
sampled_roughness,
eta);
# if 0
Is it intended for the merge?
It is mainly there to be able to validate the label, roughness and eta estimation.
this will be useful when the BSDF models change (e.g., after Principle_v2).
Do you have a better idea or suggestion?
Not really, was just curious if it is something left over from earlier development, or still intended to be useful for debugging.
One idea could be to use
#ifdef WITH_CYCLES_DEBUG
, but not really sure.I think it's fine as is, but could use a comment.
done
done
The overall logic and code organization seems reasonable.
It would be nice to do a refactor at some point after this to make a struct like
BsdfSample
and reduce the number of individual arguments and variables used in code like this.@ -532,0 +691,4 @@
risSamples[1].bsdf_pdf = surface_shader_bsdf_eval_pdfs(
kg, sd, risSamples[1].wo, &risSamples[1].bsdf_eval, unguided_bsdf_pdfs, 0);
risSamples[1].label = risSamples[0].label;
risSamples[1].avg_bsdf_eval = (risSamples[1].bsdf_eval.sum[0] +
Use
average()
Done
@Sergey I switched to the safe checks and polished the code a little bet.
It is ready for your final review now ;)
@sherholz Thanks for the update! There seems to be some out-of-order comments: we checked on the updated code and think the final round of comments is there already :)
@blender-bot package
Package build started. Download here when ready.
Only thing I would like to see change still is bumping the minimum OpenPGL version.
After that this is ok to go into main for 4.0 (in two days from now).
@brecht bumping the version is on my list.
I am pretty much waiting dependency builder and svns to be updated:
#104895
But don't you think that would fit better in a separate PR?
This would also give me the chance to clean up the code and remove older version checks separately.
When you mean 4.0 I assume we are too late for 3.6?
It's done already, all platforms have OpenPGL 0.5.
Ok, it can be done in a separate PR too.
Yes, I'd rather not make any bigger changes in 3.6 anymore this close to bcon3.
Ahh OK, I wasn't sure since there are still to points missing:
Add Package file to SVN
and
Update install_linux_packages script
I will start preparing a separate one as soon as this PR is merged.
I completely understand. I just asked so that I can forward the info to the community in the thread.
For 4.0 I will probably also cover translucent materials and have better support for equi-angular sampling.
Great, thanks.
9709cd2348
to6cce149144
Reviewers