Cycles: improve shadow terminator handling #68920

Open
opened 2019-08-20 21:30:21 +02:00 by Dalai Felinto · 76 comments

We now use a ray position start offset based on how far the smooth normal deviates from the geometry normal.

Potential further improvements:

  • Use for indirect light rays if worth the cost?
  • Use for AO rays?
We now use a ray position start offset based on how far the smooth normal deviates from the geometry normal. Potential further improvements: - [ ] Use for indirect light rays if worth the cost? - [ ] Use for AO rays?
Author
Owner

Added subscriber: @dfelinto

Added subscriber: @dfelinto

Added subscriber: @dabuxian

Added subscriber: @dabuxian

Added subscriber: @c2ba

Added subscriber: @c2ba

To address this issue, what do you think about adding a "shadow bias" parameter to control an offset from the surface along the geometric normal ? I know its not a perfect solution but at least it would allow users to avoid a lot of rendering artifacts that are due to shadow termination.

If you're ok with that I can try an implementation.

To address this issue, what do you think about adding a "shadow bias" parameter to control an offset from the surface along the geometric normal ? I know its not a perfect solution but at least it would allow users to avoid a lot of rendering artifacts that are due to shadow termination. If you're ok with that I can try an implementation.

Added subscriber: @brecht

Added subscriber: @brecht

I'd prefer us to first try a better solution, like ignoring backfaces, which may not be all that complicated.

Also as described in #37814#773025, this needs changes in BSDF evaluation too.

I'd prefer us to first try a better solution, like ignoring backfaces, which may not be all that complicated. Also as described in #37814#773025, this needs changes in BSDF evaluation too.

In #68920#794109, @brecht wrote:
I'd prefer us to first try a better solution, like ignoring backfaces, which may not be all that complicated.

Also as described in #37814#773025, this needs changes in BSDF evaluation too.

From what I understand of this "problem", this visual artifact is mainly due to low poly meshes having smooth shading enabled: you cannot fix the fact that a triangle is completely hidden from a light (from a point/directional light the artifact is obvious, it becomes less visible for large area lights). So for me it is basically just an incompatibility between the resolution of the mesh and its shading that try to simulate a smooth surface. Therefore I don't understand how ignoring backfaces could help, could you explain your idea ?

I never tried to dig deeper into that problem so I may miss something. Maybe some mathematical solution based on a reconstruction of the approximated continuous surface exists, but it seems complicated. In our previous renderer we "fixed" it with the simple shadow bias strategy that I mentioned, so that it can handle production. When the problem occured, our artists just increased the value and re-render the specific shot. However the problem was mitigated by the fact that artists used a subdivision level high enough on important assets, and almost only area lights.

That's why I am proposing the shadow "bias solution" here, but I can understand that it is not enough and cannot be considered robust. However with a default value of 0 I think it cannot hurt to offer that kind of control to the user, what do you think ?

> In #68920#794109, @brecht wrote: > I'd prefer us to first try a better solution, like ignoring backfaces, which may not be all that complicated. > > Also as described in #37814#773025, this needs changes in BSDF evaluation too. From what I understand of this "problem", this visual artifact is mainly due to low poly meshes having smooth shading enabled: you cannot fix the fact that a triangle is completely hidden from a light (from a point/directional light the artifact is obvious, it becomes less visible for large area lights). So for me it is basically just an incompatibility between the resolution of the mesh and its shading that try to simulate a smooth surface. Therefore I don't understand how ignoring backfaces could help, could you explain your idea ? I never tried to dig deeper into that problem so I may miss something. Maybe some mathematical solution based on a reconstruction of the approximated continuous surface exists, but it seems complicated. In our previous renderer we "fixed" it with the simple shadow bias strategy that I mentioned, so that it can handle production. When the problem occured, our artists just increased the value and re-render the specific shot. However the problem was mitigated by the fact that artists used a subdivision level high enough on important assets, and almost only area lights. That's why I am proposing the shadow "bias solution" here, but I can understand that it is not enough and cannot be considered robust. However with a default value of 0 I think it cannot hurt to offer that kind of control to the user, what do you think ?

Added subscriber: @3di

Added subscriber: @3di

I've just spoken to Jules Urbach from Octane, he's going to ask Lino from his Dev team to help you out.

I've just spoken to Jules Urbach from Octane, he's going to ask Lino from his Dev team to help you out.

@c2ba, I think someone should try an automatic solution before we add manual settings.

The issue happens when a reflection ray goes into the mesh, and then wrong shadow comes from when it exits. So if we ignore that exit, it may solve the most common type of problem. It's even possible to experiment with this using just a shader setup with a transparent BSDF on the backside, to see how well it works. Either way, any solution will need changes to the BSDF evaluation first.

@c2ba, I think someone should try an automatic solution before we add manual settings. The issue happens when a reflection ray goes into the mesh, and then wrong shadow comes from when it exits. So if we ignore that exit, it may solve the most common type of problem. It's even possible to experiment with this using just a shader setup with a transparent BSDF on the backside, to see how well it works. Either way, any solution will need changes to the BSDF evaluation first.

same problem with volumes too? identical model left and right

image.png

volumes shading.blend

same problem with volumes too? identical model left and right ![image.png](https://archive.blender.org/developer/F7818967/image.png) [volumes shading.blend](https://archive.blender.org/developer/F7818970/volumes_shading.blend)

In #68920#795532, @brecht wrote:
@c2ba, I think someone should try an automatic solution before we add manual settings.

The issue happens when a reflection ray goes into the mesh, and then wrong shadow comes from when it exits. So if we ignore that exit, it may solve the most common type of problem. It's even possible to experiment with this using just a shader setup with a transparent BSDF on the backside, to see how well it works. Either way, any solution will need changes to the BSDF evaluation first.

Is this what you meant Brecht? Doesn't seem to make any difference if so. Also makes no difference if shadows are disabled on the light, and if glossy rays are turned off for the light.

image.png

trying to fix with transparency on back faces.blend

> In #68920#795532, @brecht wrote: > @c2ba, I think someone should try an automatic solution before we add manual settings. > > The issue happens when a reflection ray goes into the mesh, and then wrong shadow comes from when it exits. So if we ignore that exit, it may solve the most common type of problem. It's even possible to experiment with this using just a shader setup with a transparent BSDF on the backside, to see how well it works. Either way, any solution will need changes to the BSDF evaluation first. Is this what you meant Brecht? Doesn't seem to make any difference if so. Also makes no difference if shadows are disabled on the light, and if glossy rays are turned off for the light. ![image.png](https://archive.blender.org/developer/F7819069/image.png) [trying to fix with transparency on back faces.blend](https://archive.blender.org/developer/F7819075/trying_to_fix_with_transparency_on_back_faces.blend)

This seems to be the most used solution I can find:

https://computergraphics.stackexchange.com/questions/4986/ray-tracing-shadows-the-shadow-line-artifact

and this one from siggraph this year, which I thought had already been implemented? Doesn't seem to be working if so.

https://dl.acm.org/citation.cfm?id=3328172

This seems to be the most used solution I can find: https://computergraphics.stackexchange.com/questions/4986/ray-tracing-shadows-the-shadow-line-artifact and this one from siggraph this year, which I thought had already been implemented? Doesn't seem to be working if so. https://dl.acm.org/citation.cfm?id=3328172
Member

Added subscriber: @Stefan_Werner

Added subscriber: @Stefan_Werner
Member

Read the SIGGRAPH paper more closely - it applies to problems introduced by bump/normal mapped surfaces only. It is implemented and working.

Read the SIGGRAPH paper more closely - it applies to problems introduced by bump/normal mapped surfaces only. It is implemented and working.

Added subscriber: @LukasStockner

Added subscriber: @LukasStockner

Yeah the problem of shadow terminator for normal maps seems to be fixed thanks to https://developer.blender.org/D5399

Cleary the shadows are smoothed enough where the artifact occurs.

However @LukasStockner I was wondering why you limited it to diffuse surfaces ? the paper does not mention anything about any kind of diffuse restriction, so in the cloth scene provided by Stefan if you replace the diffuse BSDF with a glossy one the artefact pops up again. Two screenshots:

So I think the fix should be generalized to any kind of scattering behavior (except maybe toon ones ?)


Now the original and "simpler" shadow terminator issue (this one https://computergraphics.stackexchange.com/questions/4986/ray-tracing-shadows-the-shadow-line-artifact) is a different problem. I don't think it can be solved without adding lighting from the backface of the triangle (in bsdf_eval function and sample probably, doing something in the else case of "if (dot(sd->Ng, omega_in) >= 0.0f) {"). That's a bit cheating but, after all, interpolated normals are a cheat. Also in my experience it is hard to do without suffering from side effects such as light leaking occuring in some cases. Maybe it should be a boolean parameter that control this behavior ?
Right now it is true that low resolution assets such as video game assets are just not acceptable when rendered with Cycles, or you have yo use large area lights to mitigate the problem :/

Yeah the problem of shadow terminator for normal maps seems to be fixed thanks to https://developer.blender.org/D5399 - Before the fix with the scene provided by Stefan: https://cloud.blender.org/p/home/5da861858ca20a9414af8cca?t - After the fix: https://cloud.blender.org/p/home/5da861fad2bf7c8344de5a5b?t Cleary the shadows are smoothed enough where the artifact occurs. However @LukasStockner I was wondering why you limited it to diffuse surfaces ? the paper does not mention anything about any kind of diffuse restriction, so in the cloth scene provided by Stefan if you replace the diffuse BSDF with a glossy one the artefact pops up again. Two screenshots: - Glossy BSDF: https://cloud.blender.org/p/home/5da862568ca20a9414af8cce?t - Fully metallic Principled: https://cloud.blender.org/p/home/5da8630dbca937ff2e5947ff?t So I think the fix should be generalized to any kind of scattering behavior (except maybe toon ones ?) --- Now the original and "simpler" shadow terminator issue (this one https://computergraphics.stackexchange.com/questions/4986/ray-tracing-shadows-the-shadow-line-artifact) is a different problem. I don't think it can be solved without adding lighting from the backface of the triangle (in bsdf_eval function and sample probably, doing something in the else case of "if (dot(sd->Ng, omega_in) >= 0.0f) {"). That's a bit cheating but, after all, interpolated normals are a cheat. Also in my experience it is hard to do without suffering from side effects such as light leaking occuring in some cases. Maybe it should be a boolean parameter that control this behavior ? Right now it is true that low resolution assets such as video game assets are just not acceptable when rendered with Cycles, or you have yo use large area lights to mitigate the problem :/
[2019-10-17 15-54-40.mp4](https://archive.blender.org/developer/F7830035/2019-10-17_15-54-40.mp4)

On the video above you can see that the shading issue is apparent even on the far right polygons where the light ray isn't passing through other faces and then hitting the back side of neighbouring faces.

2019-10-17 16-32-18.mp4

On the video above you can see that the shading issue is apparent even on the far right polygons where the light ray isn't passing through other faces and then hitting the back side of neighbouring faces. [2019-10-17 16-32-18.mp4](https://archive.blender.org/developer/F7830119/2019-10-17_16-32-18.mp4)

I'm not really sure but I think it might be related to the linear interpolation of the vertex normals on the face, that does not match the true normals a circle would have (correct me if I'm wrong). So each face get a set of interpolated normals that is slightly wrong, and since the interpolated normal is used to build the shading normal with the tangent normal map, correlation artifacts appear.
Try the following: build a cylinder in blender, remove the top and bottom faces, then in edit mode select all faces and subdivide them multiple time. Put your cylinder in smooth shading and you will see the the individual original low res faces appear, even with opengl rendering.
I guess this is because the new vertices created with subdivision have a normal computed from linear interpolation, but I'm not familiar with subdivision algorithms in general and the one used by blender in edit mode so I can only guess. Try to display the normals on edit mode with the overlay, you will see that the normals of the newly created vertices do not create a perfect circular pattern.

You have to imagine that Cycles is using theses kind of wrong normals on each point of a face. It is ok for low frequency shading, but when you add a high frequency normal map on top of that, the error becomes visible.

In brief I think your issue cannot be solved as long as interpolated normals are obtained from linear interpolation (which is the case for all renderers I can think of), you need a mesh with higher subdivisions, where normals are correctly computed (in the case of the cylinder, a catmull clark subdiv modifier gives me perfectly circular normals).

I'm not really sure but I think it might be related to the linear interpolation of the vertex normals on the face, that does not match the true normals a circle would have (correct me if I'm wrong). So each face get a set of interpolated normals that is slightly wrong, and since the interpolated normal is used to build the shading normal with the tangent normal map, correlation artifacts appear. Try the following: build a cylinder in blender, remove the top and bottom faces, then in edit mode select all faces and subdivide them multiple time. Put your cylinder in smooth shading and you will see the the individual original low res faces appear, even with opengl rendering. I guess this is because the new vertices created with subdivision have a normal computed from linear interpolation, but I'm not familiar with subdivision algorithms in general and the one used by blender in edit mode so I can only guess. Try to display the normals on edit mode with the overlay, you will see that the normals of the newly created vertices do not create a perfect circular pattern. You have to imagine that Cycles is using theses kind of wrong normals on each point of a face. It is ok for low frequency shading, but when you add a high frequency normal map on top of that, the error becomes visible. In brief I think your issue cannot be solved as long as interpolated normals are obtained from linear interpolation (which is the case for all renderers I can think of), you need a mesh with higher subdivisions, where normals are correctly computed (in the case of the cylinder, a catmull clark subdiv modifier gives me perfectly circular normals).

I've tried the same mesh in redshift and also in mantra. The shading issue is not there in either. See this bug report where I posted the comparison.

https://developer.blender.org/T70716

The test you suggested of adding extra subdivisions in edit mode should result in the shading artefacts because there's more geometry between the original edges. The same way you would intentionally add a hard edge by putting two edges close together.

The issue is still there even if using a subdivision modifier, it's just less noticeable because the angle between the faces is lower, so the difference in shading per face is lesser. There is still no smoothing of the shading between neighbouring face. Here's the same mesh with a subidivion modifier added:

image.png

I've tried the same mesh in redshift and also in mantra. The shading issue is not there in either. See this bug report where I posted the comparison. https://developer.blender.org/T70716 The test you suggested of adding extra subdivisions in edit mode should result in the shading artefacts because there's more geometry between the original edges. The same way you would intentionally add a hard edge by putting two edges close together. The issue is still there even if using a subdivision modifier, it's just less noticeable because the angle between the faces is lower, so the difference in shading per face is lesser. There is still no smoothing of the shading between neighbouring face. Here's the same mesh with a subidivion modifier added: ![image.png](https://archive.blender.org/developer/F7830480/image.png)

Here's a perfectly cylindrical mesh with perfectly cylindrical normals:

image.png

image.png

with bump map:

image.png

without bump map:

image.png

Here's the .blend if anyone wants to see if they can improve the coding to get this shading correctly. It's almost as if the bump/normal maps are using the face normal instead of the merged vertex normals for overall shading?

shading broken in cycles.blend

Here's a perfectly cylindrical mesh with perfectly cylindrical normals: ![image.png](https://archive.blender.org/developer/F7830558/image.png) ![image.png](https://archive.blender.org/developer/F7830566/image.png) with bump map: ![image.png](https://archive.blender.org/developer/F7830579/image.png) without bump map: ![image.png](https://archive.blender.org/developer/F7830582/image.png) Here's the .blend if anyone wants to see if they can improve the coding to get this shading correctly. It's almost as if the bump/normal maps are using the face normal instead of the merged vertex normals for overall shading? [shading broken in cycles.blend](https://archive.blender.org/developer/F7830585/shading_broken_in_cycles.blend)

In #68920#797692, @3di wrote:
The test you suggested of adding extra subdivisions in edit mode should result in the shading artefacts because there's more geometry between the original edges. The same way you would intentionally add a hard edge by putting two edges close together.

Ok after another test I'm convinced: while subdividing in edit mode Blender does not seem to interpolate the normal at all (all new vertex has its normal being the same as the face it was created on), so the test does not show anything sorry ^^'

I'm gonna try with Arnold tomorrow at work to see how it behaves for that scene.

I still think it might have to do with the linear interpolation of normals but again, not sure at all, some experiments in the code are required (found this old paper https://webhome.cs.uvic.ca/~blob/publications/p397-van_overveld.pdf addressing the fact that linear interpolation of normals is not a good approximation, but no mention about the interaction with normal maps).

Also the artifact is far less visible with Eevee:

Cycles:
shading broken in cycles_cycles.png

Eevee:
shading broken in cycles_eevee.png

For subdivision modifier, putting it to 3 seems to be enough to completely get rid of it:

subdiv 1:
shading broken in cycles_cycles_subdiv1.png

subdiv 2:
shading broken in cycles_cycles_subdiv2.png

subdiv 3:
shading broken in cycles_cycles_subdiv3.png

subdiv 4:
shading broken in cycles_cycles_subdiv4.png

> In #68920#797692, @3di wrote: > The test you suggested of adding extra subdivisions in edit mode should result in the shading artefacts because there's more geometry between the original edges. The same way you would intentionally add a hard edge by putting two edges close together. Ok after another test I'm convinced: while subdividing in edit mode Blender does not seem to interpolate the normal at all (all new vertex has its normal being the same as the face it was created on), so the test does not show anything sorry ^^' I'm gonna try with Arnold tomorrow at work to see how it behaves for that scene. I still think it might have to do with the linear interpolation of normals but again, not sure at all, some experiments in the code are required (found this old paper https://webhome.cs.uvic.ca/~blob/publications/p397-van_overveld.pdf addressing the fact that linear interpolation of normals is not a good approximation, but no mention about the interaction with normal maps). Also the artifact is far less visible with Eevee: Cycles: ![shading broken in cycles_cycles.png](https://archive.blender.org/developer/F7831623/shading_broken_in_cycles_cycles.png) Eevee: ![shading broken in cycles_eevee.png](https://archive.blender.org/developer/F7831621/shading_broken_in_cycles_eevee.png) For subdivision modifier, putting it to 3 seems to be enough to completely get rid of it: subdiv 1: ![shading broken in cycles_cycles_subdiv1.png](https://archive.blender.org/developer/F7831633/shading_broken_in_cycles_cycles_subdiv1.png) subdiv 2: ![shading broken in cycles_cycles_subdiv2.png](https://archive.blender.org/developer/F7831631/shading_broken_in_cycles_cycles_subdiv2.png) subdiv 3: ![shading broken in cycles_cycles_subdiv3.png](https://archive.blender.org/developer/F7831630/shading_broken_in_cycles_cycles_subdiv3.png) subdiv 4: ![shading broken in cycles_cycles_subdiv4.png](https://archive.blender.org/developer/F7831632/shading_broken_in_cycles_cycles_subdiv4.png)

Tried with Luxrender, it does not have the problem:

  • Lux:
    shading broken in cycles_lux.png
  • Cycles:
    shading broken in cycles_cycles-2.png
Tried with Luxrender, it does not have the problem: - Lux: ![shading broken in cycles_lux.png](https://archive.blender.org/developer/F7831763/shading_broken_in_cycles_lux.png) - Cycles: ![shading broken in cycles_cycles-2.png](https://archive.blender.org/developer/F7831767/shading_broken_in_cycles_cycles-2.png)

It seems strongly influenced by the normal adjustment that was introduced in: https://developer.blender.org/D2574
For example by removing the line https://developer.blender.org/differential/changeset/?ref=147520 I get the following image:
shading broken in cycles_281_no_bump_fix.png
For comparison, I have this without remove the line:
shading broken in cycles_281_bump_fix.png

Some triangles are still slightly visible on the right of the mesh, but nothing like the original.

Also it strikes me how different is the overhaul look of the mesh when the normal is not corrected according to D2574. It's like the illumination is completely different Oo

It seems strongly influenced by the normal adjustment that was introduced in: https://developer.blender.org/D2574 For example by removing the line https://developer.blender.org/differential/changeset/?ref=147520 I get the following image: ![shading broken in cycles_281_no_bump_fix.png](https://archive.blender.org/developer/F7834915/shading_broken_in_cycles_281_no_bump_fix.png) For comparison, I have this without remove the line: ![shading broken in cycles_281_bump_fix.png](https://archive.blender.org/developer/F7834918/shading_broken_in_cycles_281_bump_fix.png) Some triangles are still slightly visible on the right of the mesh, but nothing like the original. Also it strikes me how different is the overhaul look of the mesh when the normal is not corrected according to [D2574](https://archive.blender.org/developer/D2574). It's like the illumination is completely different Oo

This comment was removed by @3di

*This comment was removed by @3di*

Is there any way to vote to get this escalated to high priority? I literally can't use cycles at all in scenes with curved surfaces that have bump/normal maps......which is most scenes.

Is there any way to vote to get this escalated to high priority? I literally can't use cycles at all in scenes with curved surfaces that have bump/normal maps......which is most scenes.

No, we judge the priority, it is not determined by voting.

No, we judge the priority, it is not determined by voting.

This seems to fix it. Left is a bump map, middle is a normal map, right is a normal texture run through an alternative node setup instead of the normal map node. The bump is controlled by map range nodes instead of the normal map's strength parameter.

image.png

the normal pass:

image.png

The file:

shading issue fix.blend

This seems to fix it. Left is a bump map, middle is a normal map, right is a normal texture run through an alternative node setup instead of the normal map node. The bump is controlled by map range nodes instead of the normal map's strength parameter. ![image.png](https://archive.blender.org/developer/F7839268/image.png) the normal pass: ![image.png](https://archive.blender.org/developer/F7839271/image.png) The file: [shading issue fix.blend](https://archive.blender.org/developer/F7839273/shading_issue_fix.blend)

Slightly better version:

Same again, left is the bump map node, middle is the normal map node, right is my custom node setup, but this time mixing the object smooth normals back in.

image.png

Normal pass:

image.png

file:

shading issue fix mk2.blend

Slightly better version: Same again, left is the bump map node, middle is the normal map node, right is my custom node setup, but this time mixing the object smooth normals back in. ![image.png](https://archive.blender.org/developer/F7839405/image.png) Normal pass: ![image.png](https://archive.blender.org/developer/F7839408/image.png) file: [shading issue fix mk2.blend](https://archive.blender.org/developer/F7839414/shading_issue_fix_mk2.blend)

Also works with a bump texture:

image.png

Also works with a bump texture: ![image.png](https://archive.blender.org/developer/F7839463/image.png)

It can also be used to get rid of shading artefacts on smooth surfaces:

image.png

It can also be used to get rid of shading artefacts on smooth surfaces: ![image.png](https://archive.blender.org/developer/F7839670/image.png)
Member

For what it's worth, a little insight in what other renderers do (in the case without bump maps)
LuxRender: Nothing.
"the black triangles on the edge are unavoidable, their geometric normal points away from the light source so there is no way to fix them aside increasing tessellation and/or using softer shadows (due to a larger light source, etc.)"
https://forums.luxcorerender.org/viewtopic.php?f=5&t=1286

RedShift: Shadow ray bias derived from triangle size and the deviation between geometric and shading normal
https://docs.redshift3d.com/display/RSDOCS/Shadow+Ray+Biasing

For what it's worth, a little insight in what other renderers do (in the case without bump maps) LuxRender: Nothing. "the black triangles on the edge are unavoidable, their geometric normal points away from the light source so there is no way to fix them aside increasing tessellation and/or using softer shadows (due to a larger light source, etc.)" https://forums.luxcorerender.org/viewtopic.php?f=5&t=1286 RedShift: Shadow ray bias derived from triangle size and the deviation between geometric and shading normal https://docs.redshift3d.com/display/RSDOCS/Shadow+Ray+Biasing

In #68920#798951, @Stefan_Werner wrote:
RedShift: Shadow ray bias derived from triangle size and the deviation between geometric and shading normal
https://docs.redshift3d.com/display/RSDOCS/Shadow+Ray+Biasing

For this one to work, some change is required in bsdf evaluation because right now the contribution is completely black when dot(Ng, omega_in) < 0.

> In #68920#798951, @Stefan_Werner wrote: > RedShift: Shadow ray bias derived from triangle size and the deviation between geometric and shading normal > https://docs.redshift3d.com/display/RSDOCS/Shadow+Ray+Biasing For this one to work, some change is required in bsdf evaluation because right now the contribution is completely black when dot(Ng, omega_in) < 0.
Member

I don't think there's any way around this without ignoring backfacing geometric normals.

I don't think there's any way around this without ignoring backfacing geometric normals.

@Stefan_Werner Here's another piece of test geometry. This time the parts of the model with the artefacts aren't that low poly, also there's no normal map applied. Not sure if it's actually a bit worse than in previous builds? Currently I'm using 2.82 (sub 1), branch: master, commit date: 2019-11-22 18:14, hash: 373e936e3e

smooth shading bug test geometry.blend

image.png

@Stefan_Werner Here's another piece of test geometry. This time the parts of the model with the artefacts aren't that low poly, also there's no normal map applied. Not sure if it's actually a bit worse than in previous builds? Currently I'm using 2.82 (sub 1), branch: master, commit date: 2019-11-22 18:14, hash: `373e936e3e` [smooth shading bug test geometry.blend](https://archive.blender.org/developer/F8168837/smooth_shading_bug_test_geometry.blend) ![image.png](https://archive.blender.org/developer/F8168835/image.png)
Member

I was made aware that Appleseed has introduced a workaround for the shadow terminator. Their method, just like the fix for the bump map issue, is pushing the shadow terminator towards the light source, that is darkening the surface overall.

A quick and dirty attempt at implementing it in Cycles:
5de4c6371d

I was made aware that Appleseed has introduced a workaround for the shadow terminator. Their method, just like the fix for the bump map issue, is pushing the shadow terminator towards the light source, that is darkening the surface overall. A quick and dirty attempt at implementing it in Cycles: https://github.com/skwerner/blender/commit/5de4c6371d8f772e04e6c0383dfd8f1be6a29bd1

@Stefan_Werner , awesome! To get this, do I have to download blender from github and build myself, or is this in todays daily build from the main website? Never build it before.

@Stefan_Werner , awesome! To get this, do I have to download blender from github and build myself, or is this in todays daily build from the main website? Never build it before.

Added subscriber: @SteffenD

Added subscriber: @SteffenD
Member

You'd have to build your own, and I consider this patch nothing more than a proof of concept. In my limited testing, it appeared to break CUDA rendering and I haven't tested it BRDFs other than diffuse and principled surface.

You'd have to build your own, and I consider this patch nothing more than a proof of concept. In my limited testing, it appeared to break CUDA rendering and I haven't tested it BRDFs other than diffuse and principled surface.

I quickly tried your fix on a sphere. The parameter is animated from 0 to 1 over 100 frames. It seems to work well but need to be fine tuned to get rid of artifacts while not loosing too much lighting. I also get funny results starting from shadow_terminator_offset = 0.67 approximately :p
video0001-0100.mkv

I quickly tried your fix on a sphere. The parameter is animated from 0 to 1 over 100 frames. It seems to work well but need to be fine tuned to get rid of artifacts while not loosing too much lighting. I also get funny results starting from shadow_terminator_offset = 0.67 approximately :p [video0001-0100.mkv](https://archive.blender.org/developer/F8172881/video0001-0100.mkv)

Added subscriber: @FrancoisBeaune

Added subscriber: @FrancoisBeaune

Hi, Franz from appleseed here.

A quick and dirty attempt at implementing it in Cycles:

Awesome :)

I also get funny results starting from shadow_terminator_offset = 0.67 approximately :p

Indeed, the theorethical maximum limit before "ringing" appears is exactly 2/3 (0.666...).

Hi, Franz from appleseed here. > A quick and dirty attempt at implementing it in Cycles: Awesome :) > I also get funny results starting from shadow_terminator_offset = 0.67 approximately :p Indeed, the theorethical maximum limit before "ringing" appears is exactly 2/3 (0.666...).

I noticed that as the offset is increased, the edge of the shadow becomes harder, would this need to be compensated for in the code?

I noticed that as the offset is increased, the edge of the shadow becomes harder, would this need to be compensated for in the code?

Added subscriber: @Muritaka

Added subscriber: @Muritaka

Added subscriber: @semimetallic

Added subscriber: @semimetallic

Added subscriber: @SamGreen

Added subscriber: @SamGreen

Added subscriber: @Renderbicks

Added subscriber: @Renderbicks

Hi devs,

it would be great to keep an eye on this to find a solution. Arnold and Redshift have fixed the ray-tracing feature successfully.

The topic comes up again and again in the Blender groups on Facebook and everytime I fight hard to explain that this isn't really a bug and not ignored by the devs. People react very annoyed. Actually I could research a little workaround by offsetting the shadow to cover the Terminator effect manipulating the normals but this is more a dirty workaround for artists struggling with low-poly designs. The workaround affects also specular/reflection too extreme.

Anyways. Thanks for your overall great work.

blender_tp1YK4qBQm.png

blender_b0fJahWYWZ.png

Hi devs, it would be great to keep an eye on this to find a solution. Arnold and Redshift have fixed the ray-tracing feature successfully. The topic comes up again and again in the Blender groups on Facebook and everytime I fight hard to explain that this isn't really a bug and not ignored by the devs. People react very annoyed. Actually I could research a little workaround by offsetting the shadow to cover the Terminator effect manipulating the normals but this is more a dirty workaround for artists struggling with low-poly designs. The workaround affects also specular/reflection too extreme. Anyways. Thanks for your overall great work. ![blender_tp1YK4qBQm.png](https://archive.blender.org/developer/F8507370/blender_tp1YK4qBQm.png) ![blender_b0fJahWYWZ.png](https://archive.blender.org/developer/F8507369/blender_b0fJahWYWZ.png)

@Renderbicks , @Stefan_Werner has fixed the problem a bit higher up using the opensource code from appleseed. I think we're just awaiting someone to authorise it's incorporation into Cycles.

On the Blender Live stream @dfelinto mentioned @brecht plans to look at it in the next few months.

@Renderbicks , @Stefan_Werner has fixed the problem a bit higher up using the opensource code from appleseed. I think we're just awaiting someone to authorise it's incorporation into Cycles. On the Blender Live stream @dfelinto mentioned @brecht plans to look at it in the next few months.

Added subscriber: @John_Lancaster

Added subscriber: @John_Lancaster

I noticed that EEVEE too has a terminator problem, though it is less notable. Can the fix work for both?

I noticed that EEVEE too has a terminator problem, though it is less notable. Can the fix work for both?

Hi devs,

it would be great to keep an eye on this to find a solution. Arnold and Redshift have fixed the ray-tracing feature successfully.

The topic comes up again and again in the Blender groups on Facebook and everytime I fight hard to explain that this isn't really a bug and not ignored by the devs. People react very annoyed. Actually I could research a little workaround by offsetting the shadow to cover the Terminator effect manipulating the normals but this is more a dirty workaround for artists struggling with low-poly designs. The workaround affects also specular/reflection too extreme.

Anyways. Thanks for your overall great work.

blender_tp1YK4qBQm.png

blender_b0fJahWYWZ.png

In #68920#922422, @3di wrote:
@Renderbicks , @Stefan_Werner has fixed the problem a bit higher up using the opensource code from appleseed. I think we're just awaiting someone to authorise it's incorporation into Cycles.

On the Blender Live stream @dfelinto mentioned @brecht plans to look at it in the next few months.

I saw a comparison of an Appleseed rendering and it's using an offset of the shadow to cover the Terminator problem. If this is still the fix then it's more a workaround but not a solution because the shadow is calculated wrong in this case.

Hi devs, it would be great to keep an eye on this to find a solution. Arnold and Redshift have fixed the ray-tracing feature successfully. The topic comes up again and again in the Blender groups on Facebook and everytime I fight hard to explain that this isn't really a bug and not ignored by the devs. People react very annoyed. Actually I could research a little workaround by offsetting the shadow to cover the Terminator effect manipulating the normals but this is more a dirty workaround for artists struggling with low-poly designs. The workaround affects also specular/reflection too extreme. Anyways. Thanks for your overall great work. ![blender_tp1YK4qBQm.png](https://archive.blender.org/developer/F8507370/blender_tp1YK4qBQm.png) ![blender_b0fJahWYWZ.png](https://archive.blender.org/developer/F8507369/blender_b0fJahWYWZ.png) > In #68920#922422, @3di wrote: > @Renderbicks , @Stefan_Werner has fixed the problem a bit higher up using the opensource code from appleseed. I think we're just awaiting someone to authorise it's incorporation into Cycles. > > On the Blender Live stream @dfelinto mentioned @brecht plans to look at it in the next few months. I saw a comparison of an Appleseed rendering and it's using an offset of the shadow to cover the Terminator problem. If this is still the fix then it's more a workaround but not a solution because the shadow is calculated wrong in this case.

Hi, Franz here, appleseed founder & lead dev.

Historically our strategy to alleviate shadow handling terminator artifacts was to let the user specify a per-object shadow ray offset (we called it ray bias). That was an acceptable "solution" as a last resort but by no means a general one.

The new technique that we use (I call it "frequency-shifted cosine") is far from perfect and can cause new artifacts but it's strictly superior to our previous ray bias technique. On the upside it's very cheap at runtime and trivial to implement.

There are definitely other techniques, I'm thinking of one in particular (not my invention) but it requires to add handling code in the intersection code.

Hi, Franz here, appleseed founder & lead dev. Historically our strategy to alleviate shadow handling terminator artifacts was to let the user specify a per-object shadow ray offset (we called it ray bias). That was an acceptable "solution" as a last resort but by no means a general one. The new technique that we use (I call it "frequency-shifted cosine") is far from perfect and can cause new artifacts but it's strictly superior to our previous ray bias technique. On the upside it's very cheap at runtime and trivial to implement. There are definitely other techniques, I'm thinking of one in particular (not my invention) but it requires to add handling code in the intersection code.

I noticed that EEVEE too has a terminator problem, though it is less notable. Can the fix work for both?

I think it should? To expand somewhat on the "cosine-frequency shifting" technique I mentioned in my post just above: the idea is simply to hide the artifacts in the shadow by moving the light-to-shadow transition back (less light, more shadow).

> I noticed that EEVEE too has a terminator problem, though it is less notable. Can the fix work for both? I think it should? To expand somewhat on the "cosine-frequency shifting" technique I mentioned in my post just above: the idea is simply to hide the artifacts in the shadow by moving the light-to-shadow transition back (less light, more shadow).

We could like... Email someone at Arnold and Redshift and just ask for a basic rundown of how it's done. It's not like a big secret or anything. But who knows! Maybe one of them will be able to give the Blender team some pointers?

We could like... Email someone at Arnold and Redshift and just ask for a basic rundown of how it's done. It's not like a big secret or anything. But who knows! Maybe one of them will be able to give the Blender team some pointers?

Francois, you could be like our SPY! Since Appleseed is not a direct competitor. Then you would slip the devs a yellow envelope with the info!

Just an idea.

We really should just ask.

Francois, you could be like our SPY! Since Appleseed is not a direct competitor. Then you would slip the devs a yellow envelope with the info! Just an idea. We really should just ask.
Member

Updated patch at D7634

Updated patch at [D7634](https://archive.blender.org/developer/D7634)

Added subscriber: @Forshu-2

Added subscriber: @Forshu-2

Added subscriber: @andreas.atteneder

Added subscriber: @andreas.atteneder

Added subscriber: @kriptomik

Added subscriber: @kriptomik

Hi all,

I had an in depth look of this issue and my troubleshooting experience may help.

I think we have 2 issues here. They appear both in CPU and CUDA rendering, and none of them affects eevee.
Normal-issue-01.png
This one triggers some bad face normal values for a very limited set of triangles. Not directly related to our issue, but worth noting. If I had to make an educated guess, I would bet on a boundary condition handling issue with the first/last triangle triangle's stripe.

Normal-isseu-02.png
This issue only come from direct light. Indirect lighting seems not not affected.
This one is the one described in the thread. All triangles that do not "face the light" (negative dot product with the light source position) seen to be affected. I would rather think it's a bug to solve that a feature to add. I think that cycles is using a shortcut when processing direct diffuse light, and do not apply any lighting when the face normal dot product with light source position. This is not correct, according to me you can apply this simplification by checking if dot product with all 3 vertex normal composing the triangle is negative, otherwise you have to light your triangle accordingly.

It would be nice if someone familiar with cycle direct lighting calculation in the code could check this. (I'm not)

Thanks in advance for your attention

Hi all, I had an in depth look of this issue and my troubleshooting experience may help. I think we have 2 issues here. They appear both in CPU and CUDA rendering, and none of them affects eevee. ![Normal-issue-01.png](https://archive.blender.org/developer/F9912693/Normal-issue-01.png) This one triggers some bad face normal values for a very limited set of triangles. Not directly related to our issue, but worth noting. If I had to make an educated guess, I would bet on a boundary condition handling issue with the first/last triangle triangle's stripe. ![Normal-isseu-02.png](https://archive.blender.org/developer/F9912695/Normal-isseu-02.png) This issue only come from direct light. Indirect lighting seems not not affected. This one is the one described in the thread. All triangles that do not "face the light" (negative dot product with the light source position) seen to be affected. I would rather think it's a bug to solve that a feature to add. I think that cycles is using a shortcut when processing direct diffuse light, and do not apply any lighting when the face normal dot product with light source position. This is not correct, according to me you can apply this simplification by checking if dot product with all 3 vertex normal composing the triangle is negative, otherwise you have to light your triangle accordingly. It would be nice if someone familiar with cycle direct lighting calculation in the code could check this. (I'm not) Thanks in advance for your attention

@kriptomik EEVEE is using shadow mapping techniques and the Terminator Effect is a result of raytraced shadows.

@kriptomik EEVEE is using shadow mapping techniques and the Terminator Effect is a result of raytraced shadows.

@kriptomik if the triangle does not face the light, it means that for a closed mesh a shadow ray starting from it will intersect with another triangle. It's not a matter of changing the shading only, it's a mismatch between the smooth normal and the actual sharp geometry when tracing the shadow ray.

This paper explains it:
https://www.researchgate.net/publication/3208559_It's_really_not_a_rendering_bug_you_see

@kriptomik if the triangle does not face the light, it means that for a closed mesh a shadow ray starting from it will intersect with another triangle. It's not a matter of changing the shading only, it's a mismatch between the smooth normal and the actual sharp geometry when tracing the shadow ray. This paper explains it: https://www.researchgate.net/publication/3208559_It's_really_not_a_rendering_bug_you_see

In #68920#1137478, @brecht wrote:
@kriptomik if the triangle does not face the light, it means that for a closed mesh a shadow ray starting from it will intersect with another triangle. It's not a matter of changing the shading only, it's a mismatch between the smooth normal and the actual sharp geometry when tracing the shadow ray.

This paper explains it:
https://www.researchgate.net/publication/3208559_It's_really_not_a_rendering_bug_you_see

OK thanks a lot, good paper -> got it.
Found a nice solution explanation here . So hope is with us.

Thanks for your guidance !

> In #68920#1137478, @brecht wrote: > @kriptomik if the triangle does not face the light, it means that for a closed mesh a shadow ray starting from it will intersect with another triangle. It's not a matter of changing the shading only, it's a mismatch between the smooth normal and the actual sharp geometry when tracing the shadow ray. > > This paper explains it: > https://www.researchgate.net/publication/3208559_It's_really_not_a_rendering_bug_you_see OK thanks a lot, good paper -> got it. Found a nice solution explanation [here ](https://blog.yiningkarlli.com/2020/02/shadow-terminator-in-takua.html). So hope is with us. Thanks for your guidance !

The solution you linked is only for bump/normal mapping, for which we have similar code in Cycles. Artifacts from smooth normals on sharp geometry are something else.

The solution you linked is only for bump/normal mapping, for which we have similar code in Cycles. Artifacts from smooth normals on sharp geometry are something else.

Is there a non biased solution ? Intuitively, the one I mentioned in my first post should work but it would require to shoot 3 path per smooth triangle to compute lights interactions and have vertex normals available. I'm not even sure it's compatible with BVH and all the current path tracing algorithm but it should solve this discontinuity issue.

Is there a non biased solution ? Intuitively, the one I mentioned in my first post should work but it would require to shoot 3 path per smooth triangle to compute lights interactions and have vertex normals available. I'm not even sure it's compatible with BVH and all the current path tracing algorithm but it should solve this discontinuity issue.

I'm not sure what the exact solution is that you are proposing. What you would do with the result of doing a dot product or tracing a shadow ray for each vertex. You might find out that the triangle is susceptible to shadow terminator artifacts, but then how do you actually generate a smooth shadow terminator?

My earlier comments in this task suggest a possible solution by ignoring certain backfaces in ray intersection. Not sure which definition of "non biased" you are thinking of, but it would be closer to the ground truth than the current shadow terminator offset.

I'm not sure what the exact solution is that you are proposing. What you would do with the result of doing a dot product or tracing a shadow ray for each vertex. You might find out that the triangle is susceptible to shadow terminator artifacts, but then how do you actually generate a smooth shadow terminator? My earlier comments in this task suggest a possible solution by ignoring certain backfaces in ray intersection. Not sure which definition of "non biased" you are thinking of, but it would be closer to the ground truth than the current shadow terminator offset.

Removed subscriber: @dabuxian

Removed subscriber: @dabuxian

Sorry for the lag, but Ineeded some sleep :) I'm new here, so not familliar with all vocabulary, so don't hesitate to ask for clarification if required.

Rationale for this thinking is that by considering only the face normal you are loosing some information. Normal space withint the triangle is not constant, it's a linear interpolation of triangle's verctrices normals, described by the vectrices normals. So if any these 3 vectrices normal (boundary conditions) is facing the light, part of the triangle is facing the light even if it's normal (could be considered as an averaged value) is not. I've not done the math, but it's all linear so it's a reasonnable assumption.
Appart from the first issues I mentionned (let's forget about it ATM), normals continuity seems OK (smooth shading means that normals withint the same object island are continous), so if you just cast rays at that triangle without rejecting lights interaction based on triangle normal, the shadow termination shading should compute correctly for that triangle. Support for this assumtion is that the other triangles in the shadow terminator are shaded correctly, the only difference is that their normal face the light and their interaction with lights is computed, and not rejected.

"Non biased" mean not trying to hide the problem by a shader trick :)

If it still not clear, please let me know, I'll provide visual support.
Thanks again for your attention !

Sorry for the lag, but Ineeded some sleep :) I'm new here, so not familliar with all vocabulary, so don't hesitate to ask for clarification if required. Rationale for this thinking is that by considering only the face normal you are loosing some information. Normal space withint the triangle is not constant, it's a linear interpolation of triangle's verctrices normals, described by the vectrices normals. So if any these 3 vectrices normal (boundary conditions) is facing the light, part of the triangle is facing the light even if it's normal (could be considered as an averaged value) is not. I've not done the math, but it's all linear so it's a reasonnable assumption. Appart from the first issues I mentionned (let's forget about it ATM), normals continuity seems OK (smooth shading means that normals withint the same object island are continous), so if you just cast rays at that triangle without rejecting lights interaction based on triangle normal, the shadow termination shading should compute correctly for that triangle. Support for this assumtion is that the other triangles in the shadow terminator are shaded correctly, the only difference is that their normal face the light and their interaction with lights is computed, and not rejected. "Non biased" mean not trying to hide the problem by a shader trick :) If it still not clear, please let me know, I'll provide visual support. Thanks again for your attention !

so if you just cast rays at that triangle without rejecting lights interaction based on triangle normal

The issue is that when tracing a shadow ray from any point in the triangle, another triangle will be hit and the point will be considered shadowed. This is not happening based on any normal, it's purely based on vertex positions.

That is what needs to be addressed, and I don't think your solution does.

> so if you just cast rays at that triangle without rejecting lights interaction based on triangle normal The issue is that when tracing a shadow ray from any point in the triangle, another triangle will be hit and the point will be considered shadowed. This is not happening based on any normal, it's purely based on vertex positions. That is what needs to be addressed, and I don't think your solution does.

You're right, it'll hit a triangle adjacant to our bad guy (backface only, but I don't think cycles cares). I didn't get that. This would be correct if the triangle hit is not smooth, incorrect if it is smooth.
There is probably something to do about it too if you introduce the triangle that induce the shadowing vectrices normals into consideration but it's starting to be computationnally expensive (same rationale than before, we're discarding information we should not).
I'll think about it and let you know if I can find a similar solution.

Doh ! Smooth triangles really requires special attention to get the shadows right !

Your original post was just prefect in fact :) Sorry wasting your time...

You're right, it'll hit a triangle adjacant to our bad guy (backface only, but I don't think cycles cares). I didn't get that. This would be correct if the triangle hit is not smooth, incorrect if it is smooth. There is probably something to do about it too if you introduce the triangle that induce the shadowing vectrices normals into consideration but it's starting to be computationnally expensive (same rationale than before, we're discarding information we should not). I'll think about it and let you know if I can find a similar solution. Doh ! Smooth triangles really requires special attention to get the shadows right ! Your original post was just prefect in fact :) Sorry wasting your time...

Added subscriber: @ktdfly

Added subscriber: @ktdfly

Thanks to @ktdfly this is now much improved in 3.0.
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles#Shadow_Terminator

It's always possible to improve this further, but I will lower the relative priority for now.

Thanks to @ktdfly this is now much improved in 3.0. https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles#Shadow_Terminator It's always possible to improve this further, but I will lower the relative priority for now.
Brecht Van Lommel changed title from Reduce shadow terminator artifacts to Cycles: improve shadow terminator handling 2021-12-15 18:41:39 +01:00

In #68920#1273259, @brecht wrote:
Thanks to @ktdfly this is now much improved in 3.0.
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles#Shadow_Terminator

It's always possible to improve this further, but I will lower the relative priority for now.

Amazing and perfect job!

> In #68920#1273259, @brecht wrote: > Thanks to @ktdfly this is now much improved in 3.0. > https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycles#Shadow_Terminator > > It's always possible to improve this further, but I will lower the relative priority for now. Amazing and perfect job!
Contributor

Added subscriber: @RedMser

Added subscriber: @RedMser
Brecht Van Lommel added this to the Render & Cycles project 2023-02-07 19:08:06 +01:00
Philipp Oeser removed the
Interest
Render & Cycles
label 2023-02-09 13:59:50 +01:00
Sign in to join this conversation.
No Label
Interest
Alembic
Interest
Animation & Rigging
Interest
Asset Browser
Interest
Asset Browser Project Overview
Interest
Audio
Interest
Automated Testing
Interest
Blender Asset Bundle
Interest
BlendFile
Interest
Collada
Interest
Compatibility
Interest
Compositing
Interest
Core
Interest
Cycles
Interest
Dependency Graph
Interest
Development Management
Interest
EEVEE
Interest
EEVEE & Viewport
Interest
Freestyle
Interest
Geometry Nodes
Interest
Grease Pencil
Interest
ID Management
Interest
Images & Movies
Interest
Import Export
Interest
Line Art
Interest
Masking
Interest
Metal
Interest
Modeling
Interest
Modifiers
Interest
Motion Tracking
Interest
Nodes & Physics
Interest
OpenGL
Interest
Overlay
Interest
Overrides
Interest
Performance
Interest
Physics
Interest
Pipeline, Assets & IO
Interest
Platforms, Builds & Tests
Interest
Python API
Interest
Render & Cycles
Interest
Render Pipeline
Interest
Sculpt, Paint & Texture
Interest
Text Editor
Interest
Translations
Interest
Triaging
Interest
Undo
Interest
USD
Interest
User Interface
Interest
UV Editing
Interest
VFX & Video
Interest
Video Sequencer
Interest
Virtual Reality
Interest
Vulkan
Interest
Wayland
Interest
Workbench
Interest: X11
Legacy
Blender 2.8 Project
Legacy
Milestone 1: Basic, Local Asset Browser
Legacy
OpenGL Error
Meta
Good First Issue
Meta
Papercut
Meta
Retrospective
Meta
Security
Module
Animation & Rigging
Module
Core
Module
Development Management
Module
EEVEE & Viewport
Module
Grease Pencil
Module
Modeling
Module
Nodes & Physics
Module
Pipeline, Assets & IO
Module
Platforms, Builds & Tests
Module
Python API
Module
Render & Cycles
Module
Sculpt, Paint & Texture
Module
Triaging
Module
User Interface
Module
VFX & Video
Platform
FreeBSD
Platform
Linux
Platform
macOS
Platform
Windows
Priority
High
Priority
Low
Priority
Normal
Priority
Unbreak Now!
Status
Archived
Status
Confirmed
Status
Duplicate
Status
Needs Info from Developers
Status
Needs Information from User
Status
Needs Triage
Status
Resolved
Type
Bug
Type
Design
Type
Known Issue
Type
Patch
Type
Report
Type
To Do
No Milestone
No Assignees
17 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: blender/blender#68920
No description provided.