WIP: Cycles: Implement blue-noise dithered sampling #118479

Lukas Stockner · 2024-02-20T03:40:58+01:00

Lukas Stockner commented

2024-02-20 03:40:58 +01:00

This patch implements blue-noise dithered sampling as described by @nathanvegdahl here, which in turn is based on "Screen-Space Blue-Noise Diffusion of Monte Carlo Sampling Error via Hierarchical Ordering of Pixels".

The basic idea is simple: Instead of generating independent sequences for each pixel by scrambling them, we use a single sequence for the entire image, with each pixel getting one chunk of the samples. The ordering across pixels is determined by hierarchical scrambling of the pixel's position along a space-filling curve, which ends up being pretty much the same operation as already used for the underlying sequence.

While this initial implementation produces promising results (see below), there are a number of open points remaining:

The implementation could still be cleaned up quite a bit.
The logic for branching the sequence (e.g. for Bevel and AO) doesn't work anymore and needs to be adapted.
Notable improvements are only seen with low maximum SPP values. If we just set 1000 SPP and let adaptive sampling handle it, we don't really have any benefit. This might just be an inherent limitation, though.
As usual with RNG stuff, a lot of testing is needed in case there are any correlation issues somewhere.
In some cases, denoisers appear to not handle the different noise pattern well and confuse it for detail.

To try it out, enable debug mode for Cycles and switch the sampler to Sobol-Burley (should be renamed as part of the patch).
For now, here's the same test scene as in the linked article, rendered at 1spp:

Mode	Noisy	Denoised	Reference
Default
Blue noise

This patch implements blue-noise dithered sampling as described by @nathanvegdahl [here](https://psychopath.io/post/2022_07_24_owen_scrambling_based_dithered_blue_noise_sampling), which in turn is based on ["Screen-Space Blue-Noise Diffusion of Monte Carlo Sampling Error via Hierarchical Ordering of Pixels"](https://repository.kaust.edu.sa/items/1269ae24-2596-400b-a839-e54486033a93). The basic idea is simple: Instead of generating independent sequences for each pixel by scrambling them, we use a single sequence for the entire image, with each pixel getting one chunk of the samples. The ordering across pixels is determined by hierarchical scrambling of the pixel's position along a space-filling curve, which ends up being pretty much the same operation as already used for the underlying sequence. While this initial implementation produces promising results (see below), there are a number of open points remaining: - The implementation could still be cleaned up quite a bit. - The logic for branching the sequence (e.g. for Bevel and AO) doesn't work anymore and needs to be adapted. - Notable improvements are only seen with low **maximum** SPP values. If we just set 1000 SPP and let adaptive sampling handle it, we don't really have any benefit. This might just be an inherent limitation, though. - As usual with RNG stuff, a lot of testing is needed in case there are any correlation issues somewhere. - In some cases, denoisers appear to not handle the different noise pattern well and confuse it for detail. To try it out, enable debug mode for Cycles and switch the sampler to Sobol-Burley (should be renamed as part of the patch). For now, here's the same test scene as in the linked article, rendered at 1spp: | Mode | Noisy | Denoised | Reference | | - | - | - | - | | Default | ![noise_white.png](/attachments/5a40d4a3-0203-4292-814b-bc2f80605e01) | ![noise_white_denoised.png](/attachments/c7ea7129-3a59-47f0-8d93-f93a7697914a) | ![noise_ref.png](/attachments/cce976c5-e276-406d-84c7-4a58d9f6caa2) | | Blue noise | ![noise_blue.png](/attachments/4bf215c0-5b8c-4e0f-b929-ce4b7a831f87) | ![noise_blue_denoised.png](/attachments/f18a7fc7-e04d-4636-8310-394892f3a744) | |

noise_blue_denoised.png

487 KiB

noise_white_denoised.png

516 KiB

noise_blue.png

1.1 MiB

noise_ref.png

357 KiB

noise_white.png

1.1 MiB

👍 3 ❤️ 7 🎉 2

Lukas Stockner added the

Module

Render & Cycles

label 2024-02-20 03:40:58 +01:00

Lukas Stockner added 1 commit 2024-02-20 03:41:03 +01:00

4b20562c7f Cycles: Implement blue-noise dithered sampling

Alaska commented

2024-02-20 06:39:11 +01:00

Blue Noise dithered sampling replaces Sobol Burley sampling. The name of the sampler should be updated to reflect this, or it should be separated out as it's "own sampler" or a option.

Along with that, the Sobol Burley sampler is hidden behind a debug menu. Ideally this should be moved out of the debug menu if you want Blue Noise dithering to be accessible to the average user.

Sorry for "reviewing" minor features while you probably want feedback on the more important stuff.

Blue Noise dithered sampling replaces Sobol Burley sampling. The name of the sampler should be updated to reflect this, or it should be separated out as it's "own sampler" or a option. Along with that, the Sobol Burley sampler is hidden behind a debug menu. Ideally this should be moved out of the debug menu if you want Blue Noise dithering to be accessible to the average user. Sorry for "reviewing" minor features while you probably want feedback on the more important stuff.

Brecht Van Lommel commented

2024-02-20 12:27:22 +01:00

Results look great. I hope we can make this the default, and make it work well enough that the sampler choice can remain a debug option.

Nathan Vegdahl commented

2024-02-21 12:10:50 +01:00

Notable improvements are only seen with low maximum SPP values. If we just set 1000 SPP and let adaptive sampling handle it, we don't really have any benefit. This might just be an inherent limitation, though.

Yes, this is an inherent limitation of the technique, unfortunately. The blue noise properties only manifest when you've used all samples allocated to a pixel. I've investigated making partial sample counts also have blue noise properties with this technique, but no luck so far.

My only reservation about making this the default sampler—and the reason I haven't implemented this for Cycles already myself—is precisely because of this behavior. 4 samples per pixel with the max set to 4 is substantially different than 4 samples per pixel with the max set to 256, for example. And this could make the sampling settings counter-intuitive for users. That combined with the primary benefit of this technique being at low sample counts, it's not clear to me that the benefits of the technique will outweigh that potential confusion.

(Having said that, I of course like the technique, and have spent a substantial chunk of time working to improve it. But I'm just trying to be practical about the concrete benefits to Cycles users as the technique currently stands.)

> Notable improvements are only seen with low maximum SPP values. If we just set 1000 SPP and let adaptive sampling handle it, we don't really have any benefit. This might just be an inherent limitation, though. Yes, this is an inherent limitation of the technique, unfortunately. The blue noise properties only manifest when you've used *all* samples allocated to a pixel. I've investigated making partial sample counts also have blue noise properties with this technique, but no luck so far. My only reservation about making this the default sampler—and the reason I haven't implemented this for Cycles already myself—is precisely because of this behavior. 4 samples per pixel with the max set to 4 is *substantially* different than 4 samples per pixel with the max set to 256, for example. And this could make the sampling settings counter-intuitive for users. That combined with the primary benefit of this technique being at low sample counts, it's not clear to me that the benefits of the technique will outweigh that potential confusion. (Having said that, I of course *like* the technique, and have spent a substantial chunk of time working to improve it. But I'm just trying to be practical about the concrete benefits to Cycles users as the technique currently stands.)

Nathan Vegdahl reviewed 2024-02-21 12:42:19 +01:00

intern/cycles/kernel/sample/util.h

						
				@ -28,0 +29,4 @@

				 * Performs base-4 Owen scrambling on a reversed-bit unsigned integer.

				 *

				 * This is equivalent to the Laine-Karras permutation, but much higher

				 * quality.  See https://psychopath.io/post/2022_08_14_a_fast_hash_for_base_4_owen_scrambling

Nathan Vegdahl commented

2024-02-21 12:42:19 +01:00

I suspect this is just a copy/paste oversight, but just want to note that this bit:

This is equivalent to the Laine-Karras permutation, but much higher quality.

Is not true of the base-4 hash. It is not equivalent to the Laine-Karras permutation (which is base 2), and is also not especially high quality, as I outlined in the linked post.

I suspect this is just a copy/paste oversight, but just want to note that this bit: > This is equivalent to the Laine-Karras permutation, but much higher quality. Is not true of the base-4 hash. It is not equivalent to the Laine-Karras permutation (which is base 2), and is also not especially high quality, as I outlined in the linked post.

Brecht Van Lommel commented

2024-02-21 13:10:11 +01:00

I think it's useful even with the limitations. It seems quite reasonable for someone to set up a viewport render or or quick preview render to use e.g. 4 or 16 samples and benefit from this. It may be unintuitive, but for me it's not enough of a reason to make low sample renders more noisy than they could be.

👍 3 ❤️ 1

Alaska commented

2024-02-23 03:09:57 +01:00

I just wanted to note down some issues that has become more apparent with testing this pull request. The issues also applies to main and may need a bit of work to fix, so it may be best to deal with this in a seperate pull request.

As mentioned already, this sampling pattern works best when the max sample count, and the samples used for rendering, are the same (E.G. Set to 16 samples per pixel, and all 16 are used). Due to the current setup of the Cycles viewport, this behaviour causes some issues.

While navigating/updating the Cycles viewport, Cycles will either use 1, 2, 3, or 4 samples per pixel depending on the resolution of the viewport. However the sampling pattern being used is the one for normal viewport rendering, which usually means the sample count is incorrect for navigation, and the results you get are sub-par (E.G. Viewport is set to 1024 SSP, but while navigating, only 4 SSP are being used from that sequence). Maybe while Cycles is navigating around the viewport, it should use a lower sample count sequence to try and get that blue noise benefit?
In the Cycles viewport, the sample count, can change without viewport rendering restarting. For example, the user can set their sample count to 4 SSP, render those 4 samples, then increase it to 16, and Cycles will just render 12 SSP on top of the existing 4 SSP. This behaviour combined with how these sampling patterns work can result in low quality results. Luckily this isn't too much of an issue, as soon as the viewport rendering restarts (E.G. A camera/object moves, or a material is modified), you start from sample 1 again with the right sequence. But it's still something to consider. Maybe viewport rendering should restart whenever the sample count is changed?
The sample offset option can end up reducing the effectiveness of this technique if used improperly. For example, if someone sets their sample count to 4, then set their sample offset to a non integer multiple of 4, then they lose some of the blue noiseness of the render.

There is talk of this becoming the default sampling pattern, and other sampling patterns are left behind a debug menu. I have some questions related to this.

Sobol Burley does not support the Scrambling Distance feature. What will happen here?
- Will Tabulated Sobol with Scrambling Distance remain as a debug features? Or will it be accessible without the debug menu?
- Will the Scrambling Distance feature be removed (and Tabulated Sobol remain)? There was talk a while ago about whether or not scrambling distance is even worth it. No conclusions were made back then, but it may be something to re-discuss.
Should Sobol Burley without blue noise dithered sampling still be an option people can select? If so, should it be accessible to the end user, or remain behind a debug menu?

Some of these are more general questions, feel free to shift the discussion elsewhere.

I just wanted to note down some issues that has become more apparent with testing this pull request. The issues also applies to main and may need a bit of work to fix, so it may be best to deal with this in a seperate pull request. As mentioned already, this sampling pattern works best when the max sample count, and the samples used for rendering, are the same (E.G. Set to 16 samples per pixel, and all 16 are used). Due to the current setup of the Cycles viewport, this behaviour causes some issues. 1. While navigating/updating the Cycles viewport, Cycles will either use 1, 2, 3, or 4 samples per pixel depending on the resolution of the viewport. However the sampling pattern being used is the one for normal viewport rendering, which usually means the sample count is incorrect for navigation, and the results you get are sub-par (E.G. Viewport is set to 1024 SSP, but while navigating, only 4 SSP are being used from that sequence). Maybe while Cycles is navigating around the viewport, it should use a lower sample count sequence to try and get that blue noise benefit? 2. In the Cycles viewport, the sample count, can change without viewport rendering restarting. For example, the user can set their sample count to 4 SSP, render those 4 samples, then increase it to 16, and Cycles will just render 12 SSP on top of the existing 4 SSP. This behaviour combined with how these sampling patterns work can result in low quality results. Luckily this isn't too much of an issue, as soon as the viewport rendering restarts (E.G. A camera/object moves, or a material is modified), you start from sample 1 again with the right sequence. But it's still something to consider. Maybe viewport rendering should restart whenever the sample count is changed? 3. The `sample offset` option can end up reducing the effectiveness of this technique if used improperly. For example, if someone sets their sample count to 4, then set their sample offset to a non integer multiple of 4, then they lose some of the blue noiseness of the render. --- There is talk of this becoming the default sampling pattern, and other sampling patterns are left behind a debug menu. I have some questions related to this. 1. Sobol Burley does not support the Scrambling Distance feature. What will happen here? - Will Tabulated Sobol with Scrambling Distance remain as a debug features? Or will it be accessible without the debug menu? - Will the Scrambling Distance feature be removed (and Tabulated Sobol remain)? There was talk a while ago about whether or not scrambling distance is even worth it. No conclusions were made back then, but it may be something to re-discuss. 2. Should Sobol Burley without blue noise dithered sampling still be an option people can select? If so, should it be accessible to the end user, or remain behind a debug menu? Some of these are more general questions, feel free to shift the discussion elsewhere.

Lord Odin commented

2024-03-04 13:19:11 +01:00

First-time contributor

As a Render TD and someone who has used dithered sampling in production (Man In The High Castle, Silicon Valley) back when Lukas first implemented it in 2015-2016. I request that this be a feature exposed to the users. Hiding it or automating it serves no benefit for people who are truly trying to squeeze the performance out of cycles and hit budgets constraints.

I have production scenes that are below 32 samples, even a few below 16. It’s no easy task getting to these numbers.. these days I work in milliseconds not seconds, hitting upwards of 60FPS out of cycles in some scenarios on a single PC (Yes final renders with a frame saved to disk). Even on CPU!

I compete against unreal and other real time engines taking over the market using our beloved cycles in stock blender builds.

Let the TDs do their job, we are artists too.

Stefan Werner has a few patches with dithered sampling working with different sampling methods.

As a Render TD and someone who has used dithered sampling in production (Man In The High Castle, Silicon Valley) back when Lukas first implemented it in 2015-2016. I request that this be a feature exposed to the users. Hiding it or automating it serves no benefit for people who are truly trying to squeeze the performance out of cycles and hit budgets constraints. I have production scenes that are below 32 samples, even a few below 16. It’s no easy task getting to these numbers.. these days I work in milliseconds not seconds, hitting upwards of 60FPS out of cycles in some scenarios on a single PC (Yes final renders with a frame saved to disk). Even on CPU! I compete against unreal and other real time engines taking over the market using our beloved cycles in stock blender builds. Let the TDs do their job, we are artists too. Stefan Werner has a few patches with dithered sampling working with different sampling methods.

❤️ 2

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u blue-noise-dithered:LukasStockner-blue-noise-dithered

git checkout LukasStockner-blue-noise-dithered

Sign in to join this conversation.

No reviewers

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

WIP: Cycles: Implement blue-noise dithered sampling #118479

Checkout