Cycles: Many Lights Sampling #77889

New Issue

Under development in soc-2022-many-lights-sampling branch, implementing this paper .

Feedback thread with various bug reports and tests.blend files.

Tasks

Testing

Gather .blend files into regression test suite (Some files have been added in rBL63053)
Gather production .blend files for benchmark tests
Validate expected improvements are found in benchmark files

Implementation

Avoid computing importance for emitters twice for distant lights, use reservoir sampling instead
Better balancing with distant/background lights (see D16179, D16149 for some approaches)
Cleanup distance lights implementation to unify more with light tree
Support for light sampling inside of volumes
Make distant and background lights part of tree?
Fix zero radius distant light (mesh/shadow_terminator.blend test renders black)
Barbeshop light falloff shader node leads to noise, do better constant folding to avoid it
Add mesh light option for auto/none/front/back/both sides emission sampling
Auto exclude low emission strength triangles from tree for performance/memory?
Fix crash in spring (out of memory, avoided for now by excluding low emission)
Fix CUDA crash in wdas_cloud scene
Remove broken shadow pass with light tree (may restore later if there is a need)
Parallel tree build 23c5e06932
Workaround HIP internal compiler error

Future

Instancing support in light tree, instead of using memory for each instanced triangle
Make tree building and kernel data structures more memory efficient
Better handling for textured lights and shader nodes
- Estimate average strength for IES
- Estimate average pixel value for images
- Split textured lights into smaller primitives with baked emission?

Abandoned Ideas

Better weights for reservoir sampling in light_tree_sample, taking into account effective contribution of lights instead of uniform using the same weight for all
Computation of good PDF for MIS, a few different approaches
- Use an estimator for the PDF as is currently done (see D15983, though implementation may not be entirely correct)
- Use different PDF for MIS, as if not doing splitting (see D16126, the implementation found in the patch was very rough and had various issues, including compiler issues. It may be worth re-investigating this approach again and seeing what happens)
- Find another approach that is better than both
Find good default for splitting constant, determine if it needs to be exposed to the user

History

Previous attempts in GSoC 2020 and 2018.

Under development in `soc-2022-many-lights-sampling` branch, implementing [this paper ](https://sites.google.com/site/ckulla/home/many-lights). [Feedback thread ](https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773) with various bug reports and tests.blend files. ### Tasks Testing - [x] Gather .blend files into regression test suite (Some files have been added in rBL63053) - [x] Gather production .blend files for benchmark tests - [x] Validate expected improvements are found in benchmark files Implementation - [x] Avoid computing importance for emitters twice for distant lights, use reservoir sampling instead - [x] Better balancing with distant/background lights (see [D16179](https://archive.blender.org/developer/D16179), [D16149](https://archive.blender.org/developer/D16149) for some approaches) - [x] Cleanup distance lights implementation to unify more with light tree - [x] Support for light sampling inside of volumes - [x] Make distant and background lights part of tree? - [x] Fix zero radius distant light (mesh/shadow_terminator.blend test renders black) - [x] Barbeshop light falloff shader node leads to noise, do better constant folding to avoid it - [x] Add mesh light option for auto/none/front/back/both sides emission sampling - [x] Auto exclude low emission strength triangles from tree for performance/memory? - [x] Fix crash in spring (out of memory, avoided for now by excluding low emission) - [x] Fix CUDA crash in wdas_cloud scene - [x] Remove broken shadow pass with light tree (may restore later if there is a need) - [x] Parallel tree build 23c5e0693253516a51906588a361bb637ffd391b - [x] Workaround HIP internal compiler error Future - [x] Instancing support in light tree, instead of using memory for each instanced triangle - [ ] Make tree building and kernel data structures more memory efficient - [ ] Better handling for textured lights and shader nodes - [ ] Estimate average strength for IES - [ ] Estimate average pixel value for images - [ ] Split textured lights into smaller primitives with baked emission? Abandoned Ideas - [x] Better weights for reservoir sampling in `light_tree_sample`, taking into account effective contribution of lights instead of uniform using the same weight for all - [x] Computation of good PDF for MIS, a few different approaches * Use an estimator for the PDF as is currently done (see [D15983](https://archive.blender.org/developer/D15983), though implementation may not be entirely correct) * Use different PDF for MIS, as if not doing splitting (see [D16126](https://archive.blender.org/developer/D16126), the implementation found in the patch was very rough and had various issues, including compiler issues. It may be worth re-investigating this approach again and seeing what happens) * Find another approach that is better than both - [x] Find good default for splitting constant, determine if it needs to be exposed to the user ### History Previous attempts in [GSoC 2020 ](https://wiki.blender.org/wiki/User:Samkottler/GSoC2020/Proposal) and 2018.

Added subscriber: @samkottler-1

#68916 was marked as duplicate of this issue

Added subscriber: @DuarteRamos

Added subscriber: @bent

Added subscriber: @Alaska

Added subscriber: @SteffenD

Added subscriber: @brecht

Added subscribers: @dfelinto, @BeckersC, @PetterLundh, @filibis, @higgsas, @GuidoMedici, @lemenicier_julien, @marcog

Added subscriber: @Stefan_Werner

Removed subscriber: @higgsas

Added subscriber: @Drop

Added subscriber: @lsscpp

Added subscriber: @Emi_Martinez

Added subscriber: @EvertonSchneider

Added subscriber: @Garek

Added subscriber: @lichtwerk

Not sure if this is still relevant?
In any case, will set its subtype to TODO to get it out of the #triaging queue.

Not sure if this is still relevant? In any case, will set its subtype to TODO to get it out of the #triaging queue.

Changed status from 'Needs Triage' to: 'Confirmed'

Added subscriber: @Jebbly

Added subscriber: @Yuro

I have decided to bring a bunch of the test scenes reported in the Many Lights Sampling feedback thread here so it's all organised into one place.

Name of scene	.blend file	Description of scene and it's relevance to Many Lights Sampling development/testing
Bistro	Bistro.blend	A complex scene with lots of lights. A useful scene for verifying that Many Lights Sampling is an improvement over the old sampling approach.
Nvidia Attic	Nvidia Attic.blend	A complex scene with a few lights (I've personally found that this scene isn't that useful for development, and is instead just a test scene).
Ladder	Ladder.blend	A relatively simple scene with multiple lights.
Classroom	Classroom.zip	This scene is an example where we need "better balancing with distant/background lights"
Large radius spot light	Large Radius Spot light.blend	This scene contains a point light and a large radius high energy spot light. Spot lights with large radius are assigned incorrect importance on the edge of their cone of influence. Splitting is useful for scenes like this. (Note: In this file, the splitting threshold is set to 0.0 so you can see the issue.) This issue of "incorrect importance" also impacts large radius area lights, and lights that make use of textures.

Extra notes:
The Bistro Scene is a modified version of the "Amazon Lumberyard Bistro" scene from here: https://developer.nvidia.com/orca/amazon-lumberyard-bistro
The Nvidia Attic Scene is a modified version of the Nvidia Attic scene from here: https://developer.nvidia.com/attic-nvidia
The Ladder scene is a modified version of the "DLSC Example Scene" from here: https://luxcorerender.org/example-scenes/

I have decided to bring a bunch of the test scenes reported in the Many Lights Sampling feedback thread here so it's all organised into one place. |Name of scene|Example image|.blend file|Description of scene and it's relevance to Many Lights Sampling development/testing| | -- | -- | -- | -- | |Bistro|![Bistro.png](https://archive.blender.org/developer/F13612118/Bistro.png)|[Bistro.blend](https://archive.blender.org/developer/F13611695/Bistro.blend)|A complex scene with lots of lights. A useful scene for verifying that Many Lights Sampling is an improvement over the old sampling approach.| |Nvidia Attic|![Nvidia Attic.png](https://archive.blender.org/developer/F13612131/Nvidia_Attic.png)|[Nvidia Attic.blend](https://archive.blender.org/developer/F13611902/Nvidia_Attic.blend)|A complex scene with a few lights (I've personally found that this scene isn't that useful for development, and is instead just a test scene).| |Ladder|![Ladder.png](https://archive.blender.org/developer/F13612139/Ladder.png)|[Ladder.blend](https://archive.blender.org/developer/F13612106/Ladder.blend)|A relatively simple scene with multiple lights.| |Classroom|![classroom.png](https://archive.blender.org/developer/F13611829/classroom.png)|[Classroom.zip](https://archive.blender.org/developer/F13611863/Classroom.zip)| This scene is an example where we need "better balancing with distant/background lights"| |Large radius spot light|![Large Radius Spot light.png](https://archive.blender.org/developer/F13612022/Large_Radius_Spot_light.png)|[Large Radius Spot light.blend](https://archive.blender.org/developer/F13611898/Large_Radius_Spot_light.blend)|This scene contains a point light and a large radius high energy spot light. Spot lights with large radius are assigned incorrect importance on the edge of their cone of influence. Splitting is useful for scenes like this. (Note: In this file, the splitting threshold is set to 0.0 so you can see the issue.) This issue of "incorrect importance" also impacts large radius area lights, and lights that make use of textures.| Extra notes: The Bistro Scene is a modified version of the "Amazon Lumberyard Bistro" scene from here: https://developer.nvidia.com/orca/amazon-lumberyard-bistro The Nvidia Attic Scene is a modified version of the Nvidia Attic scene from here: https://developer.nvidia.com/attic-nvidia The Ladder scene is a modified version of the "DLSC Example Scene" from here: https://luxcorerender.org/example-scenes/

Checking the licenses of these scenes, Bistro and Attic are CC-BY 4.0. So we can potentially add those to our benchmark scenes as long as we add proper attribution. The ladder scene doesn't seem to specify a license.

For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the lib/tests/render/light/ folder, or maybe a new one.

For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the `lib/tests/render/light/` folder, or maybe a new one.

In #77889#1427169, @brecht wrote:
For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the lib/tests/render/light/ folder, or maybe a new one.

Here's a few ideas:

Idea	.blend file	Extra notes
Scene 1: Test many lights sampling with a scene with at least one of each light type	many_lights_sampling_different_light_types.blend	This is another scene where "Better balancing with distant/background lights " would be useful
Scene 2: Test many lights sampling in a scene that makes use of instanced mesh lights (Instanced meshes needed to have extra code put in place to ensure everything worked right)	many_lights_sampling_instanced_mesh_lights.blend
Scene 3: Test splitting on scenes with poor importance calculations (Like large radius spot lights)	many_lights_sampling_splitting_splitting_test.blend
Scene 4: Test the importance calculations of lights for translucent and opaque materials (With D16097, we do different importance calculations for each)	many_lights_sampling_translucent_material_light_importance.blend	There are three lights above the ground plane, one of which is inside the translucent ball. There is also a sun light shinning from above. There are three lights below the ground plane, and a sun light shinning from below. The lights below the ground plane are given an importance of `0` when sampling the opaque ground plane (when using D16097)
Scene 5: Test many lights sampling in a scene with multiple lights, some lights contribute to the scene, others have basically 0 contribution to the scene	many_lights_sampling_varying_brightness_lights.blend	The splitting threshold is set to 0 in this scene, it converges to a noise free result much faster for this case. This scene contains 10 lights. 4 have a relatively high brightness, while the other 6 are really dim.

And some high quality renders of each scene.

Scene 1: At least one of each light type	Scene 2: Instanced mesh lights	Scene 3: Spot light splitting test	Scene 4: Translucent material test	Scene 5: Varying brightness lights test

> In #77889#1427169, @brecht wrote: > For the regression test scenes, we are looking to gather various small .blend files that render 128x128 pixel images, sample count typically is between 16 and 64 samples so they render quickly. These should test the basic functioning as well as various corner cases. We'll then commit in them in the `lib/tests/render/light/` folder, or maybe a new one. Here's a few ideas: |Idea|.blend file|Extra notes| | -- | -- | -- | |Scene 1: Test many lights sampling with a scene with at least one of each light type|[many_lights_sampling_different_light_types.blend](https://archive.blender.org/developer/F13612483/many_lights_sampling_different_light_types.blend)|This is another scene where "Better balancing with distant/background lights " would be useful| |Scene 2: Test many lights sampling in a scene that makes use of instanced mesh lights (Instanced meshes needed to have extra code put in place to ensure everything worked right)|[many_lights_sampling_instanced_mesh_lights.blend](https://archive.blender.org/developer/F13612477/many_lights_sampling_instanced_mesh_lights.blend)| | |Scene 3: Test splitting on scenes with poor importance calculations (Like large radius spot lights)|[many_lights_sampling_splitting_splitting_test.blend](https://archive.blender.org/developer/F13612487/many_lights_sampling_splitting_splitting_test.blend)| | |Scene 4: Test the importance calculations of lights for translucent and opaque materials (With [D16097](https://archive.blender.org/developer/D16097), we do different importance calculations for each)|[many_lights_sampling_translucent_material_light_importance.blend](https://archive.blender.org/developer/F13612915/many_lights_sampling_translucent_material_light_importance.blend)|There are three lights above the ground plane, one of which is inside the translucent ball. There is also a sun light shinning from above. There are three lights below the ground plane, and a sun light shinning from below. The lights below the ground plane are given an importance of `0` when sampling the opaque ground plane (when using [D16097](https://archive.blender.org/developer/D16097))| |Scene 5: Test many lights sampling in a scene with multiple lights, some lights contribute to the scene, others have basically 0 contribution to the scene|[many_lights_sampling_varying_brightness_lights.blend](https://archive.blender.org/developer/F13612478/many_lights_sampling_varying_brightness_lights.blend)|The splitting threshold is set to 0 in this scene, it converges to a noise free result much faster for this case. This scene contains 10 lights. 4 have a relatively high brightness, while the other 6 are really dim.| And some high quality renders of each scene. |Scene 1: At least one of each light type|Scene 2: Instanced mesh lights|Scene 3: Spot light splitting test|Scene 4: Translucent material test|Scene 5: Varying brightness lights test| | -- | -- | -- | -- | -- | |![many_lights_sampling_different_light_types.png](https://archive.blender.org/developer/F13612481/many_lights_sampling_different_light_types.png)|![many_lights_sampling_instanced_mesh_lights.png](https://archive.blender.org/developer/F13612475/many_lights_sampling_instanced_mesh_lights.png)|![many_lights_sampling_splitting_splitting_test.png](https://archive.blender.org/developer/F13612484/many_lights_sampling_splitting_splitting_test.png)|![many_lights_sampling_translucent_material_light_importance.png](https://archive.blender.org/developer/F13612916/many_lights_sampling_translucent_material_light_importance.png)|![many_lights_sampling_varying_brightness_lights.png](https://archive.blender.org/developer/F13612479/many_lights_sampling_varying_brightness_lights.png)|

This issue was referenced by None@63053

Added subscriber: @Nurb2Kea

I just thought I would write down some of my thoughts related to improving the balance with distant/background lights:

What's causing the issue?
When deciding whether to sample a distant light or a light from the light tree, we take the importance of an approximate distant light and comparing it to the importance of the root node of the light tree. Then we use a random number to determine which light type we sample. This is an issue because we are approximating the importance of the distance lights and lights in the light tree off of approximate representations of these lights, and these can be inaccurate.

Another issue related to this is that the importance calculations for distant lights end up with relatively low importance compared to other light types for the same lighting contribution. I have provided a simple fix for this here: D16148

Here are some ideas on how we could fix it:

Calculate the sum of importance of all distant lights and the sum of importance of all lights in the light tree before making the decision on which to follow. - This is probably the best solution in terms of accuracy, but it's expensive and thus probably not worth pursuing for the final release.
Evenly split samples between distant lights and the light tree when the distant lights have an importance greater than 0. - It's simple and fast. It's just not that great in terms of image quality for certain scenes. An alternative approach could be taken. Split samples between the light tree and distant lights based on the number of each light. However, this doesn't work well in scenes with lots of lights and few distant lights. This is probably not worth pursuing.
Calculate the importance for a distant light and the importance for a light in the light tree make a decision on which light to sample based on those. - If I understand correctly, this will introduce complexity into our PDF calculations when considering MIS, similar to how splitting introduces complexity. However this approach can lead to some good results. I have created a prototype for this here: D16179
Use weighted reservoir sampling to pick a sample from the distant light group and compare it to the sample from the light tree. - Similar to the approach above, just done in a different manner resulting in potentially different/better results.
Maybe some other method.

Another thing related to distant light is that they'd benefit from something like splitting in certain situations. Like when different distant lights have different colours and brightnesses. This is a similar situation to the one described in the links below.

Scene 2 of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/102
Section "Two Different Brightness Lights Noise Issue" of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/62

I just thought I would write down some of my thoughts related to improving the balance with distant/background lights: What's causing the issue? When deciding whether to sample a distant light or a light from the light tree, we take the importance of an approximate distant light and comparing it to the importance of the root node of the light tree. Then we use a random number to determine which light type we sample. This is an issue because we are approximating the importance of the distance lights and lights in the light tree off of approximate representations of these lights, and these can be inaccurate. Another issue related to this is that the importance calculations for distant lights end up with relatively low importance compared to other light types for the same lighting contribution. I have provided a simple fix for this here: [D16148](https://archive.blender.org/developer/D16148) Here are some ideas on how we could fix it: 1. Calculate the sum of importance of all distant lights and the sum of importance of all lights in the light tree before making the decision on which to follow. - This is probably the best solution in terms of accuracy, but it's expensive and thus probably not worth pursuing for the final release. 2. Evenly split samples between distant lights and the light tree when the distant lights have an importance greater than 0. - It's simple and fast. It's just not that great in terms of image quality for certain scenes. An alternative approach could be taken. Split samples between the light tree and distant lights based on the number of each light. However, this doesn't work well in scenes with lots of lights and few distant lights. This is probably not worth pursuing. 3. Calculate the importance for a distant light and the importance for a light in the light tree make a decision on which light to sample based on those. - If I understand correctly, this will introduce complexity into our PDF calculations when considering MIS, similar to how splitting introduces complexity. However this approach can lead to some good results. I have created a prototype for this here: [D16179](https://archive.blender.org/developer/D16179) 4. Use weighted reservoir sampling to pick a sample from the distant light group and compare it to the sample from the light tree. - Similar to the approach above, just done in a different manner resulting in potentially different/better results. 5. Maybe some other method. --- Another thing related to distant light is that they'd benefit from something like splitting in certain situations. Like when different distant lights have different colours and brightnesses. This is a similar situation to the one described in the links below. >Scene 2 of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/102 >Section "Two Different Brightness Lights Noise Issue" of https://devtalk.blender.org/t/gsoc-2022-many-lights-sampling-in-cycles-x-feedback-thread/24773/62

Added subscriber: @Memento

I updated the remaining tasks. We now have an approach that no longer uses splitting, see D16315: Compute upper and lower bounds of emitter importance. Splitting is disabled due to increased noise. So splitting related improvements have been crossed out, assuming we can get the new approach working well enough for the benchmark scenes.

I updated the remaining tasks. We now have an approach that no longer uses splitting, see [D16315: Compute upper and lower bounds of emitter importance. Splitting is disabled due to increased noise](https://archive.blender.org/developer/D16315). So splitting related improvements have been crossed out, assuming we can get the new approach working well enough for the benchmark scenes.

Added subscriber: @Xianyao-Zhang

@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed.

Simple scene that benefits from splitting.blend

I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it.

many_lights_sampling_test_glass_reflection_pdf.blend

@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed. [Simple scene that benefits from splitting.blend](https://archive.blender.org/developer/F13765108/Simple_scene_that_benefits_from_splitting.blend) --- I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it. [many_lights_sampling_test_glass_reflection_pdf.blend](https://archive.blender.org/developer/F13851803/many_lights_sampling_test_glass_reflection_pdf.blend)

Another thing that was discovered when LazyDodo tried to build the Many Lights Sampling branch on the build bot was that the HIP GPU kernel for Vega fails to build.

Edit: This error appears to only occur on Linux.

Another thing that was discovered when LazyDodo tried to build the Many Lights Sampling branch on the build bot was that the HIP GPU kernel for Vega fails to build. Edit: This error appears to only occur on Linux.

Added subscriber: @weizhen

We let the should_split() function return false instead of deleting all splitting-related stuff because we are not 100% confident yet that the current approach is better, but splitting is inefficient without the support of multiple lights per shading point as in the paper & has more variance in many scenes without adjusting the reservoir weights, so we are trying other strategies. The improved weights (min + max) are only applied on leaf nodes now because it already works better on most test scenes we have, I suppose it will work fine on your scene if also applied on interior nodes. Tbh everything is heuristic and there is no optimal weight, it could take us forever just finding better weights. So currently I'd like to focus on bringing full functionality (eg. adding volume support), after that we could look into better weights & strategies for difficult scenes.

In #77889#1440598, @Alaska wrote:
@brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed.

Simple scene that benefits from splitting.blend

I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it.

many_lights_sampling_test_glass_reflection_pdf.blend

We let the `should_split()` function return false instead of deleting all splitting-related stuff because we are not 100% confident yet that the current approach is better, but splitting is inefficient without the support of multiple lights per shading point as in the paper & has more variance in many scenes without adjusting the reservoir weights, so we are trying other strategies. The improved weights (min + max) are only applied on leaf nodes now because it already works better on most test scenes we have, I suppose it will work fine on your scene if also applied on interior nodes. Tbh everything is heuristic and there is no optimal weight, it could take us forever just finding better weights. So currently I'd like to focus on bringing full functionality (eg. adding volume support), after that we could look into better weights & strategies for difficult scenes. > In #77889#1440598, @Alaska wrote: > @brecht I understand the changes made to switch to a non-splitting based Many Lights Sampling approach happened relatively recently and there may still be some work planned to further improve it. But in the meantime, splitting can still be useful for some scenes (like the scene I attached below). So with my limited understanding of the future improvements of Many Lights Sampling, I believe splitting is a feature worth while keeping until it can confidently be removed. > > [Simple scene that benefits from splitting.blend](https://archive.blender.org/developer/F13765108/Simple_scene_that_benefits_from_splitting.blend) > > --- > > I'm also not sure whether or not you want to add a test scene for the PDF calculations of rough glass when a light is in front and behind the surface and Cycles took the glossy path off the glass. I bring this up as you had to make specific changes to fix the PDF calculations for this situation and I'm not sure whether or not you want to test for it. > > [many_lights_sampling_test_glass_reflection_pdf.blend](https://archive.blender.org/developer/F13851803/many_lights_sampling_test_glass_reflection_pdf.blend)

Fix crash in spring and wdas_cloud scenes

I ran some tests on my computer to see if I could reproduce the issues and identify the cause. I was personally unable to reproduce a crash in wdas_cloud. But on Spring I got a Out of Memory error after consuming all my RAM and swap (My computer has 64GB of RAM and only a few GB of swap).
The Spring out of memory error comes from large number of light emitting triangles in the scene. These triangles come from the flower assets that are distributed over the foreground objects.

> Fix crash in spring and wdas_cloud scenes I ran some tests on my computer to see if I could reproduce the issues and identify the cause. I was personally unable to reproduce a crash in wdas_cloud. But on Spring I got a `Out of Memory` error after consuming all my RAM and swap (My computer has 64GB of RAM and only a few GB of swap). The Spring out of memory error comes from large number of light emitting triangles in the scene. These triangles come from the `flower` assets that are distributed over the foreground objects.

Yes, the light tree data structure does not support instancing (and neither did the old distribution), but it now quite a bit of memory per triangle. It should be possible to significantly reduce this with some optimization.

At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.

Yes, the light tree data structure does not support instancing (and neither did the old distribution), but it now quite a bit of memory per triangle. It should be possible to significantly reduce this with some optimization. At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.

At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source.

I have a few proposals to help with this:

Adjust the auto emission sampling heuristic to be even more conservative. At the moment if you have a mesh light with an emission strength of 1.0 and a pure colour (E.G. 100% red, 0% Green, 0% Blue), it will be excluded from direct light sampling. I have created two different versions of working around this. D16554, D16553
Take into account the surface area of the triangles when calculating if they emit enough light to be included in direct light sampling. However, this one is a bit difficult because the importance of the light based on surface area is based on the scale of everything else in the scene.

Then there's some other ideas that I think may be useful, but probably aren't useful for most people:

Add a viewlayer override. For example, add an option in the viewlayer properties tab to override the emission sampling setting for all materials.
Maybe make the auto emission sampling cutoff user controllable?

> At the same time I'm working on some changes that will make those flowers not part of the light tree by default, because they are just emitting a little bit and not worth the overhead to samples as a light source. I have a few proposals to help with this: 1. Adjust the `auto emission sampling heuristic` to be even more conservative. At the moment if you have a mesh light with an emission strength of 1.0 and a pure colour (E.G. 100% red, 0% Green, 0% Blue), it will be excluded from direct light sampling. I have created two different versions of working around this. [D16554](https://archive.blender.org/developer/D16554), [D16553](https://archive.blender.org/developer/D16553) 2. Take into account the surface area of the triangles when calculating if they emit enough light to be included in direct light sampling. However, this one is a bit difficult because the importance of the light based on surface area is based on the scale of everything else in the scene. Then there's some other ideas that I think may be useful, but probably aren't useful for most people: 1. Add a viewlayer override. For example, add an option in the viewlayer properties tab to override the `emission sampling` setting for all materials. 2. Maybe make the `auto emission sampling cutoff` user controllable?

I don't think a view layer option for emission sampling makes much sense, this seems to me really some per material or object, using the same setting for all objects in the scene doesn't seem right.

Maybe a there could be a scene wide cut-off, though I prefer not to add more obscure options.

I don't think a view layer option for emission sampling makes much sense, this seems to me really some per material or object, using the same setting for all objects in the scene doesn't seem right. Maybe a there could be a scene wide cut-off, though I prefer not to add more obscure options.

This issue was referenced by ac51d331df

This issue was referenced by ac51d331dfbdccb3ec6d4b7cb60909fe01d1a333

This issue was referenced by 396b407c7d

This issue was referenced by 396b407c7d14d2ad4fb43da196b7ea0cc5891fe6

I've merged code refactoring and the new emission sampling option into master, leaving just the light tree itself in the branch.

I'm still trying to isolate the CUDA wdas_cloud error and issue with HIP compilation on Vega. Verifying that these other changes work correctly by themselves helps isolate the problems a bit more.

I've merged code refactoring and the new emission sampling option into master, leaving just the light tree itself in the branch. I'm still trying to isolate the CUDA wdas_cloud error and issue with HIP compilation on Vega. Verifying that these other changes work correctly by themselves helps isolate the problems a bit more.

This issue was referenced by None@63110

Added subscriber: @makizar

This issue was referenced by 0731d78d00

This issue was referenced by 0731d78d0096939ad61b0c81cba888f5a1f98001

I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is.

I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master.

@weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with theta_e == 0 in the light tree, but not immediately spot the problem.

I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is. I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master. @weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with `theta_e == 0` in the light tree, but not immediately spot the problem.

In #77889#1454212, @brecht wrote:
I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is.

I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master.

@weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with theta_e == 0 in the light tree, but not immediately spot the problem.

Strange, I didn't encounter this problem in all_light_types.blend with angle = 0. Anyway, fixed that now 7d33edcf36

> In #77889#1454212, @brecht wrote: > I tried to get the shadow pass working, but found it was already working poorly in most scenes. I removed it for now, as some of the use cases have been replaced by the shadow catcher passes. We may restore it, but that would be with a different implementation, and it's not clear what even the correct way to do this is. > > I haven't been able to fix the HIP internal compile error so far, I may disable the light tree on HIP for now if I can't get it fixed tomorrow. I don't want this to block the merge to master. > > @weizhen, I noticed the mesh/shadow_terminator.blend test renders completely black. It has a distant light with angle zero, making it a bit smaller works again. I guess there is some issues with `theta_e == 0` in the light tree, but not immediately spot the problem. Strange, I didn't encounter this problem in all_light_types.blend with angle = 0. Anyway, fixed that now 7d33edcf36

This issue was referenced by ee89f213de

This issue was referenced by ee89f213de2ec810317141ed87b889981eb8c348

This issue was referenced by None@63118

This issue was referenced by 0808eaf44e

This issue was referenced by 0808eaf44e8130a902e31178848c74e4a47fb677

This issue was referenced by None@63122

This issue was referenced by 7d99c51e17

This issue was referenced by 7d99c51e17c4722949a90c5992cebd203246527f

This issue was referenced by None@63124

Would it be a good idea to switch the light tree over from a binary tree to a four wide tree? The original paper (http://www.aconty.com/pdf/many-lights-hpg2018.pdf) states that they switched to a four wide tree for these reasons:

While we have described our system as using binary trees, our practical implementation takes advantage of SIMD instructions and builds a four-wide tree. This change reduces the overall depth of the tree, which improves stratification by reducing the amount of sample stretching in Algorithm 3.

Would it be a good idea to switch the light tree over from a binary tree to a four wide tree? The original paper (http://www.aconty.com/pdf/many-lights-hpg2018.pdf) states that they switched to a four wide tree for these reasons: > While we have described our system as using binary trees, our practical implementation takes advantage of SIMD instructions and builds a four-wide tree. This change reduces the overall depth of the tree, which improves stratification by reducing the amount of sample stretching in Algorithm 3.

I took a look into improving how IES lights are handled, but couldn't figure out how to get it to work so I'll just document my findings.

From my understanding, the way we want to improve IES lights sampling with the light tree is by doing this:

Reading the IES file, figuring out the range of angles the IES texture projects out of, and the average brightness of the IES texture, then using those to manipulate the light tree approximation of the light. For example, if the IES texture projects out in a 90 degree cone, then we change the light tree approximation of the light to a 90 degree cone spot light.
Other ideas may be better pursuing.

Now, onto the information I found.

An IES texture is just a text file describing aspects of a light.
The IES format is suppose to tell the render engine which hemisphere the IES texture is placed in by listing a range of "vertical angles" (It may be more accurate then "hemipshere location", but from limited research online, it appears to be limited to hemispheres). We could use the "hemisphere location" from the IES texture to generate our spot light approximations to use in the light tree. Source: http://docs.autodesk.com/ACD/2011/ENU/filesAUG/WS73099cc142f48755f058a10f71c104f3-3b1a.htm
The IES texture is suppose to list a bunch of "vertical angles" within the hemisphere it occupies and then corresponding brightnesses for each vertical angle. If all vertical angles greater than a certain amount have a brightness of 0, then we can refine the spread of the IES light/spot light approximation in the light tree by culling those vertical angles.

I took a look into improving how IES lights are handled, but couldn't figure out how to get it to work so I'll just document my findings. From my understanding, the way we want to improve IES lights sampling with the light tree is by doing this: - Reading the IES file, figuring out the range of angles the IES texture projects out of, and the average brightness of the IES texture, then using those to manipulate the light tree approximation of the light. For example, if the IES texture projects out in a 90 degree cone, then we change the light tree approximation of the light to a 90 degree cone spot light. Other ideas may be better pursuing. Now, onto the information I found. 1. An IES texture is just a text file describing aspects of a light. 2. The IES format is suppose to tell the render engine which hemisphere the IES texture is placed in by listing a range of "vertical angles" (It may be more accurate then "hemipshere location", but from limited research online, it appears to be limited to hemispheres). We could use the "hemisphere location" from the IES texture to generate our spot light approximations to use in the light tree. Source: http://docs.autodesk.com/ACD/2011/ENU/filesAUG/WS73099cc142f48755f058a10f71c104f3-3b1a.htm 3. The IES texture is suppose to list a bunch of "vertical angles" within the hemisphere it occupies and then corresponding brightnesses for each vertical angle. If all vertical angles greater than a certain amount have a brightness of 0, then we can refine the spread of the IES light/spot light approximation in the light tree by culling those vertical angles.

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Cycles: Many Lights Sampling #77889

Tasks

History