Eevee Alpha Blend artifacts in lookdev and VR with Viewport Denoising enabled #90209

New Issue

Robert Bragg · 2021-07-26T21:26:45+02:00

Robert Bragg commented

2021-07-26 21:26:45 +02:00

System Information
Operating system: Windows 10
Graphics card: Nvidia GTX 970

Blender Version

Broken: version: 3.0.0 Alpha, branch: master (modified), commit date: 2021-07-26 13:54, hash: 3059853732
Broken: version: 2.93.1, branch: master, commit date: 2021-06-22 05:57, hash: 1b8d33b18c, type: release
Broken: version: 2.83.16, branch: master, commit date: 2021-06-15 08:46, hash: 051567553d, type: release
Worked: Unknown

Short description of error

Navigation within a viewport (such as panning / orbiting the scene) with objects that have a material with Alpha Blend enabled results in visual artifacts for those materials, seemingly regardless of the alpha channel itself (i.e. it affects materials with an alpha of one).

In my case I first saw the artifacts while using the VR mode in blender where these artifacts feel extremely exaggerated in some cases. I have an environment with foliage billboards that appear like they are slightly head-locked and they sway around a lot when moving in VR. I later realised I could reproduce similar artifacts outside of VR and then realised that disabling denoising avoided the artifacts in the editor and in VR.

It seems that Eevee's denoising introduces some kind of latency related to how the alpha blending is handled which results in temporal, visual artifacts while the viewport's perspective is changing.

I'd say the reproduction below probably doesn't represent how significant the artifacts feel in VR but I think it highlights the same underlying issue (the same two workarounds that affect the editor, also help in VR).

Exact steps for others to reproduce the error
Turn on Viewport Shading: Material Preview or Rendered

Delete cube
Add plane
Add material (default principled BSDF is fine, though in my case I'm working with emissive shaders and I have a modified build of Blender supporting equirectangular image textures that let me access their alpha channel)
Set material Base Color to an Image Texture.
Create a new image and generate a Color Grid to set as the base color. (with or without Alpha)
Set material blend mode to 'Alpha Blend'

Gyrate/pan/orbit the viewport view. (Note the artifacts are related to the viewport navigation that needs to move around, and just moving/rotating the plane itself won't show artifacts)

(For the Color Grid the artifacts mainly look like shimmering, and glints on the white letters. That's quite different to the original artifacts I saw in VR for my use case but still it seems to represent the same underlying issue.)

Under render properties, toggle Viewport Denoising and perform the same viewport movements to compare.
With Viewport Denoising back on, switch to using Alpha Clip instead of Alpha Blend and again compare.

eevee-denoise-artifacts.blend

**System Information** Operating system: Windows 10 Graphics card: Nvidia GTX 970 **Blender Version** Broken: version: 3.0.0 Alpha, branch: master (modified), commit date: 2021-07-26 13:54, hash: 3059853732d7 Broken: version: 2.93.1, branch: master, commit date: 2021-06-22 05:57, hash: 1b8d33b18c2f, type: release Broken: version: 2.83.16, branch: master, commit date: 2021-06-15 08:46, hash: 051567553d29, type: release Worked: Unknown **Short description of error** Navigation within a viewport (such as panning / orbiting the scene) with objects that have a material with Alpha Blend enabled results in visual artifacts for those materials, seemingly regardless of the alpha channel itself (i.e. it affects materials with an alpha of one). In my case I first saw the artifacts while using the VR mode in blender where these artifacts feel extremely exaggerated in some cases. I have an environment with foliage billboards that appear like they are slightly head-locked and they sway around a lot when moving in VR. I later realised I could reproduce similar artifacts outside of VR and then realised that disabling denoising avoided the artifacts in the editor and in VR. It seems that Eevee's denoising introduces some kind of latency related to how the alpha blending is handled which results in temporal, visual artifacts while the viewport's perspective is changing. I'd say the reproduction below probably doesn't represent how significant the artifacts feel in VR but I think it highlights the same underlying issue (the same two workarounds that affect the editor, also help in VR). **Exact steps for others to reproduce the error** Turn on Viewport Shading: Material Preview or Rendered Delete cube Add plane Add material (default principled BSDF is fine, though in my case I'm working with emissive shaders and I have a modified build of Blender supporting equirectangular image textures that let me access their alpha channel) Set material Base Color to an Image Texture. Create a new image and generate a Color Grid to set as the base color. (with or without Alpha) Set material blend mode to 'Alpha Blend' Gyrate/pan/orbit the viewport view. (Note the artifacts are related to the viewport navigation that needs to move around, and just moving/rotating the plane itself won't show artifacts) (For the Color Grid the artifacts mainly look like shimmering, and glints on the white letters. That's quite different to the original artifacts I saw in VR for my use case but still it seems to represent the same underlying issue.) Under render properties, toggle Viewport Denoising and perform the same viewport movements to compare. With Viewport Denoising back on, switch to using Alpha Clip instead of Alpha Blend and again compare. [eevee-denoise-artifacts.blend](https://archive.blender.org/developer/F10239289/eevee-denoise-artifacts.blend)

Robert Bragg commented

2021-07-26 21:26:45 +02:00

Added subscriber: @RobertBragg

Robert Bragg commented

2021-07-26 21:29:55 +02:00

2021-07-26 18-20-25.mp4

[2021-07-26 18-20-25.mp4](https://archive.blender.org/developer/F10239219/2021-07-26_18-20-25.mp4)

Robert Bragg commented

2021-07-26 22:02:12 +02:00

A minimal reproduction - as described above: eevee-denoise-artifacts.blend

A minimal reproduction - as described above: [eevee-denoise-artifacts.blend](https://archive.blender.org/developer/F10239289/eevee-denoise-artifacts.blend)

Philipp Oeser commented

2021-08-05 11:51:59 +02:00

Added subscribers: @fclem, @lichtwerk

Philipp Oeser commented

2021-08-05 11:51:59 +02:00

Changed status from 'Needs Triage' to: 'Needs Developer To Reproduce'

Philipp Oeser commented

2021-08-05 11:51:59 +02:00

Since temporal reprojection can leave some ghosting, I would say that this is a limitation one has to live with? (@fclem: correct me if I ma wrong -- it is kind of drastic with Alpha Blend for sure...)

Robert Bragg commented

2021-08-05 23:44:22 +02:00

To add a bit... With the original artifacts I saw in VR that led me to investigate this (which are difficult to capture and show here) I'd say it's not really a ghosting that's the main problem in VR - it's more like a reprojection artifact where transparent geometry gets moved but with the wrong depth associated with it. In VR this makes it look like transparent geometry is physically swimming around in space, not in a fixed location - so it's not just as if there is a ghost trail or blurring.

When I first saw this issue in VR it was with a billboard texture with some foliage for my environment and as I was looking at it and moving my head around is was like the whole thing was physically swaying around (not with a blur/trail, it just looked like the object itself was moving around as I moved)

I did investigate Eevee's source code to try and understand a bit more... I didn't know before that Eevee's 'Viewport Denoising' is based on temporal antialiasing, by jittering the projection matrix every frame and then blending a sequence of older 'sample' framebuffers together.

My best guess currently about what's happening here is that transparent materials interfere with a reprojection that's done as part of the temporal anti aliasing that depends on the depth of past pixels to reproject older frames into the current frame. I didn't go deep enough into the code to be 100% sure of this but it would make sense to me from what I did see. So since alpha blended/transparent pixels wouldn't write out a depth value I then imagine that those pixels end up being reprojected based on the depth of whatever is behind them.

With a 2D viewport this temporary artifact while moving the camera is probably not that big of a deal. In VR though it's not really usable if you have things in your scene that use blending since all these objects will be continuously wobbling around as they are continuously reprojected according to your head movement but based on the depth of whatever is behind them.

So just to clarify, that's mainly just my hypothesis, based on an initial skim of the code.

If that is what's happening then maybe in VR the viewport denoising could just be done in terms of rendering N samples per frame so that there's no need for any temporal reprojection ever in VR. I guess that could be a relatively minimal change with Eevee's current rendering code. Maybe in the short term viewport denoising could be automatically disabled in VR, while the current approach isn't really suited to situations where the view camera is moving continuously?

To add a bit... With the original artifacts I saw in VR that led me to investigate this (which are difficult to capture and show here) I'd say it's not really a ghosting that's the main problem in VR - it's more like a reprojection artifact where transparent geometry gets moved but with the wrong depth associated with it. In VR this makes it look like transparent geometry is physically swimming around in space, not in a fixed location - so it's not just as if there is a ghost trail or blurring. When I first saw this issue in VR it was with a billboard texture with some foliage for my environment and as I was looking at it and moving my head around is was like the whole thing was physically swaying around (not with a blur/trail, it just looked like the object itself was moving around as I moved) I did investigate Eevee's source code to try and understand a bit more... I didn't know before that Eevee's 'Viewport Denoising' is based on temporal antialiasing, by jittering the projection matrix every frame and then blending a sequence of older 'sample' framebuffers together. My best guess currently about what's happening here is that transparent materials interfere with a reprojection that's done as part of the temporal anti aliasing that depends on the depth of past pixels to reproject older frames into the current frame. I didn't go deep enough into the code to be 100% sure of this but it would make sense to me from what I did see. So since alpha blended/transparent pixels wouldn't write out a depth value I then imagine that those pixels end up being reprojected based on the depth of whatever is behind them. With a 2D viewport this temporary artifact while moving the camera is probably not that big of a deal. In VR though it's not really usable if you have things in your scene that use blending since all these objects will be continuously wobbling around as they are continuously reprojected according to your head movement but based on the depth of whatever is behind them. So just to clarify, that's mainly just my hypothesis, based on an initial skim of the code. If that is what's happening then maybe in VR the viewport denoising could just be done in terms of rendering N samples per frame so that there's no need for any temporal reprojection ever in VR. I guess that could be a relatively minimal change with Eevee's current rendering code. Maybe in the short term viewport denoising could be automatically disabled in VR, while the current approach isn't really suited to situations where the view camera is moving continuously?

Jacob Merrill commented

2021-08-06 01:00:25 +02:00

Added subscriber: @JacobMerrill-1

Jacob Merrill commented

2021-08-06 01:00:25 +02:00

(edit: some of these effects can be solved here)

this is an effect of only getting 1 sample per frame before TAA resets,

upbge has a patch by youle to render the frame multiple times in a frame to get more samples.
https://github.com/UPBGE/upbge/pull/1474

this allows for stable soft shadows in motion as well as much better alpha hash.

there is another issue with eevee in general - https://developer.blender.org/T88331

(edit: some of these effects can be solved here) this is an effect of only getting 1 sample per frame before TAA resets, upbge has a patch by youle to render the frame multiple times in a frame to get more samples. https://github.com/UPBGE/upbge/pull/1474 this allows for stable soft shadows in motion as well as much better alpha hash. there is another issue with eevee in general - https://developer.blender.org/T88331

Clément Foucault commented

2021-10-15 19:37:08 +02:00

Changed status from 'Needs Developer To Reproduce' to: 'Confirmed'

Clément Foucault commented

2021-10-15 19:37:08 +02:00

@RobertBragg You are correct in your assumptions. Thats why most game / realtime engines turned to forward shading pipelines and MSAA for their AA solution (because it is very fast). This is not practical for us as you would loose most of the effects of eevee.

Doing 2x rendering is going to be a huge problem performance wise. But I agree we could implicitly disable Viewport Denoising at least for VR.

We could also switch to something like SMAA, however this is not the same in terms of work to do.

@RobertBragg You are correct in your assumptions. Thats why most game / realtime engines turned to forward shading pipelines and MSAA for their AA solution (because it is very fast). This is not practical for us as you would loose most of the effects of eevee. Doing 2x rendering is going to be a huge problem performance wise. But I agree we could implicitly disable Viewport Denoising at least for VR. We could also switch to something like SMAA, however this is not the same in terms of work to do.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Eevee Alpha Blend artifacts in lookdev and VR with Viewport Denoising enabled #90209