Greater difference between min and max samples introduces swirly artefacts to denoised renders. #99135

New Issue

michael campbell · 2022-06-24T15:07:19+02:00

michael campbell commented

2022-06-24 15:07:19 +02:00

System Information
Operating system: Windows-10-10.0.19044-SP0 64 Bits
Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96

Blender Version
Broken: version: 3.3.0 Alpha, branch: master, commit date: 2022-05-30 17:30, hash: ca59391704
Worked: (newest version of Blender that worked as expected)

Short description of error
Brecht asked that I raise this bug after speaking about it on the blender.chat.

Open below file which contains two identical scenes with different sample settings.
worse results higher samples.blend

on the low samples scene go to the compositor and press f12

Now go to the high sample scene's compositor and press f12

Generally the higher the difference between min and max samples, the more black swirly lines appear after denoising:

Usually setting the min samples to be >= 1% of max samples removes the issue, but occasionally (such as this example scene) the problem persists until the difference between the two is much smaller.

**System Information** Operating system: Windows-10-10.0.19044-SP0 64 Bits Graphics card: NVIDIA GeForce GTX 1070/PCIe/SSE2 NVIDIA Corporation 4.5.0 NVIDIA 512.96 **Blender Version** Broken: version: 3.3.0 Alpha, branch: master, commit date: 2022-05-30 17:30, hash: `ca59391704` Worked: (newest version of Blender that worked as expected) **Short description of error** Brecht asked that I raise this bug after speaking about it on the blender.chat. Open below file which contains two identical scenes with different sample settings. [worse results higher samples.blend](https://archive.blender.org/developer/F13224448/worse_results_higher_samples.blend) on the low samples scene go to the compositor and press f12 ![image.png](https://archive.blender.org/developer/F13224425/image.png) Now go to the high sample scene's compositor and press f12 ![image.png](https://archive.blender.org/developer/F13224436/image.png) Generally the higher the difference between min and max samples, the more black swirly lines appear after denoising: ![image.png](https://archive.blender.org/developer/F13224441/image.png) Usually setting the min samples to be >= 1% of max samples removes the issue, but occasionally (such as this example scene) the problem persists until the difference between the two is much smaller.

michael campbell commented

2022-06-24 15:07:19 +02:00

Added subscriber: @3di

Raimund Klink commented

2022-06-24 16:09:25 +02:00

Added subscriber: @Raimund58

Brecht Van Lommel commented

2022-06-24 16:15:55 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2022-06-24 16:15:55 +02:00

I'll merge this in #74581 (Cycles: adaptive sampling improvements), since it's basically a known limitation that can be improved with better algorithms, though not fully solved.

What happens is that with very few samples, it can't detect that some pixels are actually noisy and it stops sampling them after 4 samples. Meanwhile other pixels do get more samples and get less noisy. Then you get a pattern that doesn't look like noise to the denoising algorithm, but rather actual image features that it should preserve.

With a wider radius or smarter re-activation of pixels this could be improved.

I'll merge this in #74581 (Cycles: adaptive sampling improvements), since it's basically a known limitation that can be improved with better algorithms, though not fully solved. What happens is that with very few samples, it can't detect that some pixels are actually noisy and it stops sampling them after 4 samples. Meanwhile other pixels do get more samples and get less noisy. Then you get a pattern that doesn't look like noise to the denoising algorithm, but rather actual image features that it should preserve. With a wider radius or smarter re-activation of pixels this could be improved.

Brecht Van Lommel commented

2022-06-24 16:16:07 +02:00

Closed as duplicate of #74581

Brecht Van Lommel closed this issue

2022-06-24 16:16:07 +02:00

michael campbell commented

2022-06-24 16:29:08 +02:00

@brecht Thanks. What would be considered too few samples? I was getting this earlier at 60 min 600 max, but not at 40 min and 600 max. The main culprit appears to be the noise threshold being set lower in the image with the artefacts:

max 600, min 40, threshold 0.1

max 600, min 60, threshold 0.05

@brecht Thanks. What would be considered too few samples? I was getting this earlier at 60 min 600 max, but not at 40 min and 600 max. The main culprit appears to be the noise threshold being set lower in the image with the artefacts: max 600, min 40, threshold 0.1 ![image.png](https://archive.blender.org/developer/F13224659/image.png) max 600, min 60, threshold 0.05 ![image.png](https://archive.blender.org/developer/F13224682/image.png)

Brecht Van Lommel commented

2022-06-24 16:31:32 +02:00

It depends on the scene and settings, if the noisy image has structured patterns rather than more uniform noise there will be problems.

michael campbell commented

2022-06-24 16:35:55 +02:00

Actually, I don't think the known issue you mentioned can be the cause of this particular issue, because lowering the max samples and leaving the min samples the same also stops the artefacts appearing. For example in this scene, if you lower the max samples to something like 50, then the artefacts also disappear.

michael campbell commented

2022-06-24 16:37:38 +02:00

I'll create another scene to demonstrate the issue with much higher min samples, and include a scene where lowering the max samples will remove the artefacts.

Brecht Van Lommel commented

2022-06-24 16:54:00 +02:00

It's probably a variation of the same issues though.

michael campbell commented

2022-06-24 17:04:41 +02:00

Fair enough. At least you now know that the issue isn't only caused by too low samples, otherwise the train station scene above would have had the swirls on the 40 min samples rather than the 60 min samples. It appears to be caused by too large of a range between min and max, and get's amplified the lower the noise level is set (I wasn't aware of that previously, but the attached scene seems to demonstrate that).

michael campbell commented

2022-06-24 18:40:17 +02:00

Hi @brecht, not sure if this will help you, but here's another example scene for you to try which indicates this report doesn't appear to be related to insufficient min samples. This time identical min samples produce a different result just by lowering the noise threshold.

Steps:

open below file
go to 'scene high noise threshold' scene's compositor and hit render (it has 0.1 noise threshold)
now go to 'scene low noise threshld (it has 0.05 noise threshold but all other sample settings are identical (min 10, max 200000))

Can you think of any reason why lowering the noise threshold would introduce artefacts with identical min/max samples? Do you think there could be an equation to calculate the required min to max ratio dependant on the noise threshold?

Here's the scene:
basic test.rar

Hi @brecht, not sure if this will help you, but here's another example scene for you to try which indicates this report doesn't appear to be related to insufficient min samples. This time identical min samples produce a different result just by lowering the noise threshold. Steps: 1) open below file 2) go to 'scene high noise threshold' scene's compositor and hit render (it has 0.1 noise threshold) ![image.png](https://archive.blender.org/developer/F13225436/image.png) 3) now go to 'scene low noise threshld (it has 0.05 noise threshold but all other sample settings are identical (min 10, max 200000)) ![image.png](https://archive.blender.org/developer/F13225432/image.png) Can you think of any reason why lowering the noise threshold would introduce artefacts with identical min/max samples? Do you think there could be an equation to calculate the required min to max ratio dependant on the noise threshold? Here's the scene: [basic test.rar](https://archive.blender.org/developer/F13225447/basic_test.rar)

Brecht Van Lommel commented

2022-06-29 17:03:45 +02:00

It's still a variation of the same issue. The higher threshold is so noisy that the denoiser determines that it must be noise. The lower threshold makes it less noisy, which makes the denoiser think it's an actual texture pattern instead of noise. AI denoisers are unpredictable like that.

Fundamentally I think the issue is still the uneven noise distribution, that needs to be addressed with solutions like those planned in #74581 (Cycles: adaptive sampling improvements).

It's still a variation of the same issue. The higher threshold is so noisy that the denoiser determines that it must be noise. The lower threshold makes it less noisy, which makes the denoiser think it's an actual texture pattern instead of noise. AI denoisers are unpredictable like that. Fundamentally I think the issue is still the uneven noise distribution, that needs to be addressed with solutions like those planned in #74581 (Cycles: adaptive sampling improvements).

michael campbell commented

2022-06-29 18:37:08 +02:00

Isn't the noise level of the pixel the same at 10 samples regardless of whether the threshold is 0.1 or 0.01? I don't understand why in the example kitchen scene above that the same pixels would be considered clean at 10 samples if the threshold is set to 0.05 (so stops rendering that pixel), but is classed as noise at 10 samples when set to 0.1 threshold (so keeps rendering it), causing the clear pattern between incorrectly ended pixels and cleaned pixels that the AI sees as intentional. The pixel must have an identical noise level at 10 samples, so what in the code is deciding the noise level is less than 0.05 but not less than 0.1. I've downloaded and built a debug version, so if you can offer any clues that would be great.

Brecht Van Lommel commented

2022-06-29 18:46:19 +02:00

Some pixels are black and get no further samples at all in either case, since 10 samples was not enough detect anything in them. For other pixels, they will get more samples with 0.05 than with 0.1, and so those pixels will have different levels of noise.

That means for 0.1 you end up with a situation with very noisy pixels and black pixels, and the AI denoiser considers the whole thing to be noisy. Where as with 0.05 you end with with a combination of less noisy pixels and black pixels, and the AI denoiser considers that some kind of pattern it should preserve.

Some pixels are black and get no further samples at all in either case, since 10 samples was not enough detect anything in them. For other pixels, they will get more samples with 0.05 than with 0.1, and so those pixels will have different levels of noise. That means for 0.1 you end up with a situation with very noisy pixels and black pixels, and the AI denoiser considers the whole thing to be noisy. Where as with 0.05 you end with with a combination of less noisy pixels and black pixels, and the AI denoiser considers that some kind of pattern it should preserve.

michael campbell commented

2022-06-29 19:00:48 +02:00

I'm struggling to apply that to the example I gave you the other day on blender chat, where it was happening with (min samples set to 1440 and max samples at 14440 with noise level .01), but not at (min samples 36, max samples 352 noise threshold 0.1).

Anyway, I'll stop pestering you. I was just keen to try and get some insight to help me understand what's happening under the hood, because my addon includes various sample pre-set which this issue is interfering with: https://3dillusions.gumroad.com/l/turbo_tools

I'm struggling to apply that to the example I gave you the other day on blender chat, where it was happening with (min samples set to 1440 and max samples at 14440 with noise level .01), but not at (min samples 36, max samples 352 noise threshold 0.1). Anyway, I'll stop pestering you. I was just keen to try and get some insight to help me understand what's happening under the hood, because my addon includes various sample pre-set which this issue is interfering with: https://3dillusions.gumroad.com/l/turbo_tools

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Greater difference between min and max samples introduces swirly artefacts to denoised renders. #99135