Cycles: Many Lights Sampling #77889

Open
opened 2020-06-15 16:52:52 +02:00 by Sam Kottler · 54 comments
Member

Under development in soc-2022-many-lights-sampling branch, implementing this paper .

Feedback thread with various bug reports and tests.blend files.

Tasks

Testing

  • Gather .blend files into regression test suite (Some files have been added in rBL63053)
  • Gather production .blend files for benchmark tests
  • Validate expected improvements are found in benchmark files

Implementation

  • Avoid computing importance for emitters twice for distant lights, use reservoir sampling instead

  • Better balancing with distant/background lights (see D16179, D16149 for some approaches)

  • Cleanup distance lights implementation to unify more with light tree

  • Support for light sampling inside of volumes

  • Make distant and background lights part of tree?

  • Fix zero radius distant light (mesh/shadow_terminator.blend test renders black)

  • Barbeshop light falloff shader node leads to noise, do better constant folding to avoid it

  • Add mesh light option for auto/none/front/back/both sides emission sampling

  • Auto exclude low emission strength triangles from tree for performance/memory?

  • Fix crash in spring (out of memory, avoided for now by excluding low emission)

  • Fix CUDA crash in wdas_cloud scene

  • Remove broken shadow pass with light tree (may restore later if there is a need)

  • Parallel tree build 23c5e06932

  • Workaround HIP internal compiler error

Future

  • Instancing support in light tree, instead of using memory for each instanced triangle
  • Make tree building and kernel data structures more memory efficient
  • Better handling for textured lights and shader nodes
    • Estimate average strength for IES
    • Estimate average pixel value for images
    • Split textured lights into smaller primitives with baked emission?

Abandoned Ideas

  • Better weights for reservoir sampling in light_tree_sample, taking into account effective contribution of lights instead of uniform using the same weight for all
  • Computation of good PDF for MIS, a few different approaches
    • Use an estimator for the PDF as is currently done (see D15983, though implementation may not be entirely correct)
    • Use different PDF for MIS, as if not doing splitting (see D16126, the implementation found in the patch was very rough and had various issues, including compiler issues. It may be worth re-investigating this approach again and seeing what happens)
    • Find another approach that is better than both
  • Find good default for splitting constant, determine if it needs to be exposed to the user

History

Previous attempts in GSoC 2020 and 2018.

Under development in `soc-2022-many-lights-sampling` branch, implementing [this paper ](https://sites.google.com/site/ckulla/home/many-lights). [Feedback thread ](https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773) with various bug reports and tests.blend files. ### Tasks Testing - [x] Gather .blend files into regression test suite (Some files have been added in rBL63053) - [x] Gather production .blend files for benchmark tests - [x] Validate expected improvements are found in benchmark files Implementation - [x] Avoid computing importance for emitters twice for distant lights, use reservoir sampling instead - [x] Better balancing with distant/background lights (see [D16179](https://archive.blender.org/developer/D16179), [D16149](https://archive.blender.org/developer/D16149) for some approaches) - [x] Cleanup distance lights implementation to unify more with light tree - [x] Support for light sampling inside of volumes - [x] Make distant and background lights part of tree? - [x] Fix zero radius distant light (mesh/shadow_terminator.blend test renders black) - [x] Barbeshop light falloff shader node leads to noise, do better constant folding to avoid it - [x] Add mesh light option for auto/none/front/back/both sides emission sampling - [x] Auto exclude low emission strength triangles from tree for performance/memory? - [x] Fix crash in spring (out of memory, avoided for now by excluding low emission) - [x] Fix CUDA crash in wdas_cloud scene - [x] Remove broken shadow pass with light tree (may restore later if there is a need) - [x] Parallel tree build 23c5e0693253516a51906588a361bb637ffd391b - [x] Workaround HIP internal compiler error Future - [x] Instancing support in light tree, instead of using memory for each instanced triangle - [ ] Make tree building and kernel data structures more memory efficient - [ ] Better handling for textured lights and shader nodes - [ ] Estimate average strength for IES - [ ] Estimate average pixel value for images - [ ] Split textured lights into smaller primitives with baked emission? Abandoned Ideas - [x] Better weights for reservoir sampling in `light_tree_sample`, taking into account effective contribution of lights instead of uniform using the same weight for all - [x] Computation of good PDF for MIS, a few different approaches * Use an estimator for the PDF as is currently done (see [D15983](https://archive.blender.org/developer/D15983), though implementation may not be entirely correct) * Use different PDF for MIS, as if not doing splitting (see [D16126](https://archive.blender.org/developer/D16126), the implementation found in the patch was very rough and had various issues, including compiler issues. It may be worth re-investigating this approach again and seeing what happens) * Find another approach that is better than both - [x] Find good default for splitting constant, determine if it needs to be exposed to the user ### History Previous attempts in [GSoC 2020 ](https://wiki.blender.org/wiki/User:Samkottler/GSoC2020/Proposal) and 2018.
Sam Kottler self-assigned this 2020-06-15 16:52:52 +02:00
Author
Member

Added subscriber: @samkottler-1

Added subscriber: @samkottler-1

#68916 was marked as duplicate of this issue

#68916 was marked as duplicate of this issue

Added subscriber: @DuarteRamos

Added subscriber: @DuarteRamos

Added subscriber: @bent

Added subscriber: @bent
Member

Added subscriber: @Alaska

Added subscriber: @Alaska

Added subscriber: @SteffenD

Added subscriber: @SteffenD

Added subscriber: @brecht

Added subscriber: @brecht
Added subscribers: @dfelinto, @BeckersC, @PetterLundh, @filibis, @higgsas, @GuidoMedici, @lemenicier_julien, @marcog
Member

Added subscriber: @Stefan_Werner

Added subscriber: @Stefan_Werner

Removed subscriber: @higgsas

Removed subscriber: @higgsas

Added subscriber: @Drop

Added subscriber: @Drop

Added subscriber: @lsscpp

Added subscriber: @lsscpp

Added subscriber: @Emi_Martinez

Added subscriber: @Emi_Martinez

Added subscriber: @EvertonSchneider

Added subscriber: @EvertonSchneider

Added subscriber: @Garek

Added subscriber: @Garek
Member

Added subscriber: @lichtwerk

Added subscriber: @lichtwerk
Member

Not sure if this is still relevant?
In any case, will set its subtype to TODO to get it out of the #triaging queue.

Not sure if this is still relevant? In any case, will set its subtype to TODO to get it out of the #triaging queue.
Member

Changed status from 'Needs Triage' to: 'Confirmed'

Changed status from 'Needs Triage' to: 'Confirmed'
Sam Kottler was unassigned by Brecht Van Lommel 2022-07-25 18:25:47 +02:00
Member

Added subscriber: @Jebbly

Added subscriber: @Jebbly

Added subscriber: @Yuro

Added subscriber: @Yuro
Member

I have decided to bring a bunch of the test scenes reported in the Many Lights Sampling feedback thread here so it's all organised into one place.

Name of scene Example image .blend file Description of scene and it's relevance to Many Lights Sampling development/testing
Bistro Bistro.png Bistro.blend A complex scene with lots of lights. A useful scene for verifying that Many Lights Sampling is an improvement over the old sampling approach.
Nvidia Attic Nvidia Attic.png Nvidia Attic.blend A complex scene with a few lights (I've personally found that this scene isn't that useful for development, and is instead just a test scene).
Ladder Ladder.png Ladder.blend A relatively simple scene with multiple lights.
Classroom classroom.png Classroom.zip This scene is an example where we need "better balancing with distant/background lights"
Large radius spot light Large Radius Spot light.png Large Radius Spot light.blend This scene contains a point light and a large radius high energy spot light. Spot lights with large radius are assigned incorrect importance on the edge of their cone of influence. Splitting is useful for scenes like this. (Note: In this file, the splitting threshold is set to 0.0 so you can see the issue.) This issue of "incorrect importance" also impacts large radius area lights, and lights that make use of textures.

Extra notes:
The Bistro Scene is a modified version of the "Amazon Lumberyard Bistro" scene from here: https://developer.nvidia.com/orca/amazon-lumberyard-bistro
The Nvidia Attic Scene is a modified version of the Nvidia Attic scene from here: https://developer.nvidia.com/attic-nvidia
The Ladder scene is a modified version of the "DLSC Example Scene" from here: https://luxcorerender.org/example-scenes/

I have decided to bring a bunch of the test scenes reported in the Many Lights Sampling feedback thread here so it's all organised into one place. |Name of scene|Example image|.blend file|Description of scene and it's relevance to Many Lights Sampling development/testing| | -- | -- | -- | -- | |Bistro|![Bistro.png](https://archive.blender.org/developer/F13612118/Bistro.png)|[Bistro.blend](https://archive.blender.org/developer/F13611695/Bistro.blend)|A complex scene with lots of lights. A useful scene for verifying that Many Lights Sampling is an improvement over the old sampling approach.| |Nvidia Attic|![Nvidia Attic.png](https://archive.blender.org/developer/F13612131/Nvidia_Attic.png)|[Nvidia Attic.blend](https://archive.blender.org/developer/F13611902/Nvidia_Attic.blend)|A complex scene with a few lights (I've personally found that this scene isn't that useful for development, and is instead just a test scene).| |Ladder|![Ladder.png](https://archive.blender.org/developer/F13612139/Ladder.png)|[Ladder.blend](https://archive.blender.org/developer/F13612106/Ladder.blend)|A relatively simple scene with multiple lights.| |Classroom|![classroom.png](https://archive.blender.org/developer/F13611829/classroom.png)|[Classroom.zip](https://archive.blender.org/developer/F13611863/Classroom.zip)| This scene is an example where we need "better balancing with distant/background lights"| |Large radius spot light|![Large Radius Spot light.png](https://archive.blender.org/developer/F13612022/Large_Radius_Spot_light.png)|[Large Radius Spot light.blend](https://archive.blender.org/developer/F13611898/Large_Radius_Spot_light.blend)|This scene contains a point light and a large radius high energy spot light. Spot lights with large radius are assigned incorrect importance on the edge of their cone of influence. Splitting is useful for scenes like this. (Note: In this file, the splitting threshold is set to 0.0 so you can see the issue.) This issue of "incorrect importance" also impacts large radius area lights, and lights that make use of textures.| Extra notes: The Bistro Scene is a modified version of the "Amazon Lumberyard Bistro" scene from here: https://developer.nvidia.com/orca/amazon-lumberyard-bistro The Nvidia Attic Scene is a modified version of the Nvidia Attic scene from here: https://developer.nvidia.com/attic-nvidia The Ladder scene is a modified version of the "DLSC Example Scene" from here: https://luxcorerender.org/example-scenes/

Checking the licenses of these scenes, Bistro and Attic are CC-BY 4.0. So we can potentially add those to our benchmark scenes as long as we add proper attribution. The ladder scene doesn't seem to specify a license.

Checking the licenses of these scenes, Bistro and Attic are CC-BY 4.0. So we can potentially add those to our benchmark scenes as long as we add proper attribution. The ladder scene doesn't seem to specify a license.

For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the lib/tests/render/light/ folder, or maybe a new one.

For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the `lib/tests/render/light/` folder, or maybe a new one.
Member

In #77889#1427169, @brecht wrote:
For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the lib/tests/render/light/ folder, or maybe a new one.

Here's a few ideas:

Idea .blend file Extra notes
Scene 1: Test many lights sampling with a scene with at least one of each light type many_lights_sampling_different_light_types.blend This is another scene where "Better balancing with distant/background lights " would be useful
Scene 2: Test many lights sampling in a scene that makes use of instanced mesh lights (Instanced meshes needed to have extra code put in place to ensure everything worked right) many_lights_sampling_instanced_mesh_lights.blend
Scene 3: Test splitting on scenes with poor importance calculations (Like large radius spot lights) many_lights_sampling_splitting_splitting_test.blend
Scene 4: Test the importance calculations of lights for translucent and opaque materials (With D16097, we do different importance calculations for each) many_lights_sampling_translucent_material_light_importance.blend There are three lights above the ground plane, one of which is inside the translucent ball. There is also a sun light shinning from above. There are three lights below the ground plane, and a sun light shinning from below. The lights below the ground plane are given an importance of 0 when sampling the opaque ground plane (when using D16097)
Scene 5: Test many lights sampling in a scene with multiple lights, some lights contribute to the scene, others have basically 0 contribution to the scene many_lights_sampling_varying_brightness_lights.blend The splitting threshold is set to 0 in this scene, it converges to a noise free result much faster for this case. This scene contains 10 lights. 4 have a relatively high brightness, while the other 6 are really dim.

And some high quality renders of each scene.

Scene 1: At least one of each light type Scene 2: Instanced mesh lights Scene 3: Spot light splitting test Scene 4: Translucent material test Scene 5: Varying brightness lights test
many_lights_sampling_different_light_types.png many_lights_sampling_instanced_mesh_lights.png many_lights_sampling_splitting_splitting_test.png many_lights_sampling_translucent_material_light_importance.png many_lights_sampling_varying_brightness_lights.png
> In #77889#1427169, @brecht wrote: > For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the `lib/tests/render/light/` folder, or maybe a new one. Here's a few ideas: |Idea|.blend file|Extra notes| | -- | -- | -- | |Scene 1: Test many lights sampling with a scene with at least one of each light type|[many_lights_sampling_different_light_types.blend](https://archive.blender.org/developer/F13612483/many_lights_sampling_different_light_types.blend)|This is another scene where "Better balancing with distant/background lights " would be useful| |Scene 2: Test many lights sampling in a scene that makes use of instanced mesh lights (Instanced meshes needed to have extra code put in place to ensure everything worked right)|[many_lights_sampling_instanced_mesh_lights.blend](https://archive.blender.org/developer/F13612477/many_lights_sampling_instanced_mesh_lights.blend)| | |Scene 3: Test splitting on scenes with poor importance calculations (Like large radius spot lights)|[many_lights_sampling_splitting_splitting_test.blend](https://archive.blender.org/developer/F13612487/many_lights_sampling_splitting_splitting_test.blend)| | |Scene 4: Test the importance calculations of lights for translucent and opaque materials (With [D16097](https://archive.blender.org/developer/D16097), we do different importance calculations for each)|[many_lights_sampling_translucent_material_light_importance.blend](https://archive.blender.org/developer/F13612915/many_lights_sampling_translucent_material_light_importance.blend)|There are three lights above the ground plane, one of which is inside the translucent ball. There is also a sun light shinning from above. There are three lights below the ground plane, and a sun light shinning from below. The lights below the ground plane are given an importance of `0` when sampling the opaque ground plane (when using [D16097](https://archive.blender.org/developer/D16097))| |Scene 5: Test many lights sampling in a scene with multiple lights, some lights contribute to the scene, others have basically 0 contribution to the scene|[many_lights_sampling_varying_brightness_lights.blend](https://archive.blender.org/developer/F13612478/many_lights_sampling_varying_brightness_lights.blend)|The splitting threshold is set to 0 in this scene, it converges to a noise free result much faster for this case. This scene contains 10 lights. 4 have a relatively high brightness, while the other 6 are really dim.| And some high quality renders of each scene. |Scene 1: At least one of each light type|Scene 2: Instanced mesh lights|Scene 3: Spot light splitting test|Scene 4: Translucent material test|Scene 5: Varying brightness lights test| | -- | -- | -- | -- | -- | |![many_lights_sampling_different_light_types.png](https://archive.blender.org/developer/F13612481/many_lights_sampling_different_light_types.png)|![many_lights_sampling_instanced_mesh_lights.png](https://archive.blender.org/developer/F13612475/many_lights_sampling_instanced_mesh_lights.png)|![many_lights_sampling_splitting_splitting_test.png](https://archive.blender.org/developer/F13612484/many_lights_sampling_splitting_splitting_test.png)|![many_lights_sampling_translucent_material_light_importance.png](https://archive.blender.org/developer/F13612916/many_lights_sampling_translucent_material_light_importance.png)|![many_lights_sampling_varying_brightness_lights.png](https://archive.blender.org/developer/F13612479/many_lights_sampling_varying_brightness_lights.png)|

This issue was referenced by None@63053

This issue was referenced by None@63053

Added subscriber: @Nurb2Kea

Added subscriber: @Nurb2Kea
Member

I just thought I would write down some of my thoughts related to improving the balance with distant/background lights:

What's causing the issue?
When deciding whether to sample a distant light or a light from the light tree, we take the importance of an approximate distant light and comparing it to the importance of the root node of the light tree. Then we use a random number to determine which light type we sample. This is an issue because we are approximating the importance of the distance lights and lights in the light tree off of approximate representations of these lights, and these can be inaccurate.

Another issue related to this is that the importance calculations for distant lights end up with relatively low importance compared to other light types for the same lighting contribution. I have provided a simple fix for this here: D16148

Here are some ideas on how we could fix it:

  1. Calculate the sum of importance of all distant lights and the sum of importance of all lights in the light tree before making the decision on which to follow. - This is probably the best solution in terms of accuracy, but it's expensive and thus probably not worth pursuing for the final release.

  2. Evenly split samples between distant lights and the light tree when the distant lights have an importance greater than 0. - It's simple and fast. It's just not that great in terms of image quality for certain scenes. An alternative approach could be taken. Split samples between the light tree and distant lights based on the number of each light. However, this doesn't work well in scenes with lots of lights and few distant lights. This is probably not worth pursuing.

  3. Calculate the importance for a distant light and the importance for a light in the light tree make a decision on which light to sample based on those. - If I understand correctly, this will introduce complexity into our PDF calculations when considering MIS, similar to how splitting introduces complexity. However this approach can lead to some good results. I have created a prototype for this here: D16179

  4. Use weighted reservoir sampling to pick a sample from the distant light group and compare it to the sample from the light tree. - Similar to the approach above, just done in a different manner resulting in potentially different/better results.

  5. Maybe some other method.


Another thing related to distant light is that they'd benefit from something like splitting in certain situations. Like when different distant lights have different colours and brightnesses. This is a similar situation to the one described in the links below.

Scene 2 of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/102
Section "Two Different Brightness Lights Noise Issue" of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/62

I just thought I would write down some of my thoughts related to improving the balance with distant/background lights: What's causing the issue? When deciding whether to sample a distant light or a light from the light tree, we take the importance of an approximate distant light and comparing it to the importance of the root node of the light tree. Then we use a random number to determine which light type we sample. This is an issue because we are approximating the importance of the distance lights and lights in the light tree off of approximate representations of these lights, and these can be inaccurate. Another issue related to this is that the importance calculations for distant lights end up with relatively low importance compared to other light types for the same lighting contribution. I have provided a simple fix for this here: [D16148](https://archive.blender.org/developer/D16148) Here are some ideas on how we could fix it: 1. Calculate the sum of importance of all distant lights and the sum of importance of all lights in the light tree before making the decision on which to follow. - This is probably the best solution in terms of accuracy, but it's expensive and thus probably not worth pursuing for the final release. 2. Evenly split samples between distant lights and the light tree when the distant lights have an importance greater than 0. - It's simple and fast. It's just not that great in terms of image quality for certain scenes. An alternative approach could be taken. Split samples between the light tree and distant lights based on the number of each light. However, this doesn't work well in scenes with lots of lights and few distant lights. This is probably not worth pursuing. 3. Calculate the importance for a distant light and the importance for a light in the light tree make a decision on which light to sample based on those. - If I understand correctly, this will introduce complexity into our PDF calculations when considering MIS, similar to how splitting introduces complexity. However this approach can lead to some good results. I have created a prototype for this here: [D16179](https://archive.blender.org/developer/D16179) 4. Use weighted reservoir sampling to pick a sample from the distant light group and compare it to the sample from the light tree. - Similar to the approach above, just done in a different manner resulting in potentially different/better results. 5. Maybe some other method. --- Another thing related to distant light is that they'd benefit from something like splitting in certain situations. Like when different distant lights have different colours and brightnesses. This is a similar situation to the one described in the links below. >Scene 2 of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/102 >Section "Two Different Brightness Lights Noise Issue" of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/62

Added subscriber: @Memento

Added subscriber: @Memento

I updated the remaining tasks. We now have an approach that no longer uses splitting, see D16315: Compute upper and lower bounds of emitter importance. Splitting is disabled due to increased noise. So splitting related improvements have been crossed out, assuming we can get the new approach working well enough for the benchmark scenes.

I updated the remaining tasks. We now have an approach that no longer uses splitting, see [D16315: Compute upper and lower bounds of emitter importance. Splitting is disabled due to increased noise](https://archive.blender.org/developer/D16315). So splitting related improvements have been crossed out, assuming we can get the new approach working well enough for the benchmark scenes.

Added subscriber: @Xianyao-Zhang

Added subscriber: @Xianyao-Zhang
Member

@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed.

Simple scene that benefits from splitting.blend


I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it.

many_lights_sampling_test_glass_reflection_pdf.blend

@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed. [Simple scene that benefits from splitting.blend](https://archive.blender.org/developer/F13765108/Simple_scene_that_benefits_from_splitting.blend) --- I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it. [many_lights_sampling_test_glass_reflection_pdf.blend](https://archive.blender.org/developer/F13851803/many_lights_sampling_test_glass_reflection_pdf.blend)
Member

Another thing that was discovered when LazyDodo tried to build the Many Lights Sampling branch on the build bot was that the HIP GPU kernel for Vega fails to build.

Edit: This error appears to only occur on Linux.

Another thing that was discovered when LazyDodo tried to build the Many Lights Sampling branch on the build bot was that the HIP GPU kernel for Vega fails to build. Edit: This error appears to only occur on Linux.
Member

Added subscriber: @weizhen

Added subscriber: @weizhen
Member

We let the should_split() function return false instead of deleting all splitting-related stuff because we are not 100% confident yet that the current approach is better, but splitting is inefficient without the support of multiple lights per shading point as in the paper & has more variance in many scenes without adjusting the reservoir weights, so we are trying other strategies. The improved weights (min + max) are only applied on leaf nodes now because it already works better on most test scenes we have, I suppose it will work fine on your scene if also applied on interior nodes. Tbh everything is heuristic and there is no optimal weight, it could take us forever just finding better weights. So currently I'd like to focus on bringing full functionality (eg. adding volume support), after that we could look into better weights & strategies for difficult scenes.

In #77889#1440598, @Alaska wrote:
@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed.

Simple scene that benefits from splitting.blend


I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it.

many_lights_sampling_test_glass_reflection_pdf.blend

We let the `should_split()` function return false instead of deleting all splitting-related stuff because we are not 100% confident yet that the current approach is better, but splitting is inefficient without the support of multiple lights per shading point as in the paper & has more variance in many scenes without adjusting the reservoir weights, so we are trying other strategies. The improved weights (min + max) are only applied on leaf nodes now because it already works better on most test scenes we have, I suppose it will work fine on your scene if also applied on interior nodes. Tbh everything is heuristic and there is no optimal weight, it could take us forever just finding better weights. So currently I'd like to focus on bringing full functionality (eg. adding volume support), after that we could look into better weights & strategies for difficult scenes. > In #77889#1440598, @Alaska wrote: > @brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed. > > [Simple scene that benefits from splitting.blend](https://archive.blender.org/developer/F13765108/Simple_scene_that_benefits_from_splitting.blend) > > --- > > I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it. > > [many_lights_sampling_test_glass_reflection_pdf.blend](https://archive.blender.org/developer/F13851803/many_lights_sampling_test_glass_reflection_pdf.blend)
Member

Fix crash in spring and wdas_cloud scenes

I ran some tests on my computer to see if I could reproduce the issues and identify the cause. I was personally unable to reproduce a crash in wdas_cloud. But on Spring I got a Out of Memory error after consuming all my RAM and swap (My computer has 64GB of RAM and only a few GB of swap).
The Spring out of memory error comes from large number of light emitting triangles in the scene. These triangles come from the flower assets that are distributed over the foreground objects.

> Fix crash in spring and wdas_cloud scenes I ran some tests on my computer to see if I could reproduce the issues and identify the cause. I was personally unable to reproduce a crash in wdas_cloud. But on Spring I got a `Out of Memory` error after consuming all my RAM and swap (My computer has 64GB of RAM and only a few GB of swap). The Spring out of memory error comes from large number of light emitting triangles in the scene. These triangles come from the `flower` assets that are distributed over the foreground objects.

Yes, the light tree data structure does not support instancing (and neither did the old distribution), but it now quite a bit of memory per triangle. It should be possible to significantly reduce this with some optimization.

At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.

Yes, the light tree data structure does not support instancing (and neither did the old distribution), but it now quite a bit of memory per triangle. It should be possible to significantly reduce this with some optimization. At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.
Member

At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.

I have a few proposals to help with this:

  1. Adjust the auto emission sampling heuristic to be even more conservative. At the moment if you have a mesh light with an emission strength of 1.0 and a pure colour (E.G. 100% red, 0% Green, 0% Blue), it will be excluded from direct light sampling. I have created two different versions of working around this. D16554, D16553
  2. Take into account the surface area of the triangles when calculating if they emit enough light to be included in direct light sampling. However, this one is a bit difficult because the importance of the light based on surface area is based on the scale of everything else in the scene.

Then there's some other ideas that I think may be useful, but probably aren't useful for most people:

  1. Add a viewlayer override. For example, add an option in the viewlayer properties tab to override the emission sampling setting for all materials.
  2. Maybe make the auto emission sampling cutoff user controllable?
> At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source. I have a few proposals to help with this: 1. Adjust the `auto emission sampling heuristic` to be even more conservative. At the moment if you have a mesh light with an emission strength of 1.0 and a pure colour (E.G. 100% red, 0% Green, 0% Blue), it will be excluded from direct light sampling. I have created two different versions of working around this. [D16554](https://archive.blender.org/developer/D16554), [D16553](https://archive.blender.org/developer/D16553) 2. Take into account the surface area of the triangles when calculating if they emit enough light to be included in direct light sampling. However, this one is a bit difficult because the importance of the light based on surface area is based on the scale of everything else in the scene. Then there's some other ideas that I think may be useful, but probably aren't useful for most people: 1. Add a viewlayer override. For example, add an option in the viewlayer properties tab to override the `emission sampling` setting for all materials. 2. Maybe make the `auto emission sampling cutoff` user controllable?

I don't think a view layer option for emission sampling makes much sense, this seems to me really some per material or object, using the same setting for all objects in the scene doesn't seem right.

Maybe a there could be a scene wide cut-off, though I prefer not to add more obscure options.

I don't think a view layer option for emission sampling makes much sense, this seems to me really some per material or object, using the same setting for all objects in the scene doesn't seem right. Maybe a there could be a scene wide cut-off, though I prefer not to add more obscure options.

This issue was referenced by ac51d331df

This issue was referenced by ac51d331dfbdccb3ec6d4b7cb60909fe01d1a333

This issue was referenced by 396b407c7d

This issue was referenced by 396b407c7d14d2ad4fb43da196b7ea0cc5891fe6

I've merged code refactoring and the new emission sampling option into master, leaving just the light tree itself in the branch.

I'm still trying to isolate the CUDA wdas_cloud error and issue with HIP compilation on Vega. Verifying that these other changes work correctly by themselves helps isolate the problems a bit more.

I've merged code refactoring and the new emission sampling option into master, leaving just the light tree itself in the branch. I'm still trying to isolate the CUDA wdas_cloud error and issue with HIP compilation on Vega. Verifying that these other changes work correctly by themselves helps isolate the problems a bit more.

This issue was referenced by None@63110

This issue was referenced by None@63110

Added subscriber: @makizar

Added subscriber: @makizar

This issue was referenced by 0731d78d00

This issue was referenced by 0731d78d0096939ad61b0c81cba888f5a1f98001

I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is.

I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master.

@weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with theta_e == 0 in the light tree, but not immediately spot the problem.

I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is. I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master. @weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with `theta_e == 0` in the light tree, but not immediately spot the problem.
Member

In #77889#1454212, @brecht wrote:
I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is.

I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master.

@weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with theta_e == 0 in the light tree, but not immediately spot the problem.

Strange, I didn't encounter this problem in all_light_types.blend with angle = 0. Anyway, fixed that now 7d33edcf36

> In #77889#1454212, @brecht wrote: > I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is. > > I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master. > > @weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with `theta_e == 0` in the light tree, but not immediately spot the problem. Strange, I didn't encounter this problem in all_light_types.blend with angle = 0. Anyway, fixed that now 7d33edcf36

This issue was referenced by ee89f213de

This issue was referenced by ee89f213de2ec810317141ed87b889981eb8c348

This issue was referenced by None@63118

This issue was referenced by None@63118

This issue was referenced by 0808eaf44e

This issue was referenced by 0808eaf44e8130a902e31178848c74e4a47fb677

This issue was referenced by None@63122

This issue was referenced by None@63122

This issue was referenced by 7d99c51e17

This issue was referenced by 7d99c51e17c4722949a90c5992cebd203246527f

This issue was referenced by None@63124

This issue was referenced by None@63124
Member

Would it be a good idea to switch the light tree over from a binary tree to a four wide tree? The original paper (http://www.aconty.com/pdf/many-lights-hpg2018.pdf) states that they switched to a four wide tree for these reasons:

While we have described our system as using binary trees, our practical implementation takes advantage of SIMD instructions and builds a four-wide tree. This change reduces the overall depth of the tree, which improves stratification by reducing the amount of sample stretching in Algorithm 3.

Would it be a good idea to switch the light tree over from a binary tree to a four wide tree? The original paper (http://www.aconty.com/pdf/many-lights-hpg2018.pdf) states that they switched to a four wide tree for these reasons: > While we have described our system as using binary trees, our practical implementation takes advantage of SIMD instructions and builds a four-wide tree. This change reduces the overall depth of the tree, which improves stratification by reducing the amount of sample stretching in Algorithm 3.
Member

I took a look into improving how IES lights are handled, but couldn't figure out how to get it to work so I'll just document my findings.

From my understanding, the way we want to improve IES lights sampling with the light tree is by doing this:

  • Reading the IES file, figuring out the range of angles the IES texture projects out of, and the average brightness of the IES texture, then using those to manipulate the light tree approximation of the light. For example, if the IES texture projects out in a 90 degree cone, then we change the light tree approximation of the light to a 90 degree cone spot light.
    Other ideas may be better pursuing.

Now, onto the information I found.

  1. An IES texture is just a text file describing aspects of a light.
  2. The IES format is suppose to tell the render engine which hemisphere the IES texture is placed in by listing a range of "vertical angles" (It may be more accurate then "hemipshere location", but from limited research online, it appears to be limited to hemispheres). We could use the "hemisphere location" from the IES texture to generate our spot light approximations to use in the light tree. Source: http://docs.autodesk.com/ACD/2011/ENU/filesAUG/WS73099cc142f48755f058a10f71c104f3-3b1a.htm
  3. The IES texture is suppose to list a bunch of "vertical angles" within the hemisphere it occupies and then corresponding brightnesses for each vertical angle. If all vertical angles greater than a certain amount have a brightness of 0, then we can refine the spread of the IES light/spot light approximation in the light tree by culling those vertical angles.
I took a look into improving how IES lights are handled, but couldn't figure out how to get it to work so I'll just document my findings. From my understanding, the way we want to improve IES lights sampling with the light tree is by doing this: - Reading the IES file, figuring out the range of angles the IES texture projects out of, and the average brightness of the IES texture, then using those to manipulate the light tree approximation of the light. For example, if the IES texture projects out in a 90 degree cone, then we change the light tree approximation of the light to a 90 degree cone spot light. Other ideas may be better pursuing. Now, onto the information I found. 1. An IES texture is just a text file describing aspects of a light. 2. The IES format is suppose to tell the render engine which hemisphere the IES texture is placed in by listing a range of "vertical angles" (It may be more accurate then "hemipshere location", but from limited research online, it appears to be limited to hemispheres). We could use the "hemisphere location" from the IES texture to generate our spot light approximations to use in the light tree. Source: http://docs.autodesk.com/ACD/2011/ENU/filesAUG/WS73099cc142f48755f058a10f71c104f3-3b1a.htm 3. The IES texture is suppose to list a bunch of "vertical angles" within the hemisphere it occupies and then corresponding brightnesses for each vertical angle. If all vertical angles greater than a certain amount have a brightness of 0, then we can refine the spread of the IES light/spot light approximation in the light tree by culling those vertical angles.
Brecht Van Lommel added this to the Render & Cycles project 2023-02-07 19:08:06 +01:00
Philipp Oeser removed the
Interest
Render & Cycles
label 2023-02-09 14:02:05 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
21 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#77889
No description provided.