Cycles: Embree improvements #73778

New Issue

Brecht Van Lommel · 2020-02-13T14:51:30+01:00

Brecht Van Lommel commented

2020-02-13 14:51:30 +01:00

Performance

Verify performance and memory usage is close to existing CPU render
Disable compact BVH or make option (worse performance, only little memory usage reduction)
Investigate reducing memory usage by deduplicating Cycles and Embree geometry
Fix Warning: pthread_setaffinity_np failed to set affinity to thread -1 printed in console on Linux (c7329da14b)

Platform

Fix Leaking of exports into the blender binary on windows though dllexports when building it as a static lib ( 4155f8dc21 / rBL62441 )

Differences in Render Tests

D6575: Cycles: Enable quaterion motion blur in Embree.
multi_step_motion_blur.blend not showing motion blur
ambient_occlusion_only_local.blend crash
Shadow catcher test failures
Hair motion blur tests broken after change to catmull-rom
Line segments are rendered as ribbons, which breaks principled hair and other tests

CYCLESTEST_ARGS="--python-expr \"import bpy; bpy.context.scene.cycles.use_bvh_embree = True\"" ctest -R cycles

**Performance** - [x] Verify performance and memory usage is close to existing CPU render - [x] Disable compact BVH or make option (worse performance, only little memory usage reduction) - [ ] Investigate reducing memory usage by deduplicating Cycles and Embree geometry - [x] Fix `Warning: pthread_setaffinity_np failed to set affinity to thread -1` printed in console on Linux (c7329da14b) **Platform** - [x] Fix Leaking of exports into the blender binary on windows though dllexports when building it as a static lib ( 4155f8dc21 / rBL62441 ) **Differences in Render Tests** - [x] [D6575: Cycles: Enable quaterion motion blur in Embree.](https://archive.blender.org/developer/D6575) - [x] `multi_step_motion_blur.blend` not showing motion blur - [x] `ambient_occlusion_only_local.blend` crash - [x] Shadow catcher test failures - [x] Hair motion blur tests broken after change to catmull-rom - [x] Line segments are rendered as ribbons, which breaks principled hair and other tests ``` CYCLESTEST_ARGS="--python-expr \"import bpy; bpy.context.scene.cycles.use_bvh_embree = True\"" ctest -R cycles ```

Brecht Van Lommel commented

2020-02-13 14:51:30 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Brecht Van Lommel commented

2020-02-13 14:51:30 +01:00

Added subscriber: @brecht

Brecht Van Lommel commented

2020-02-13 14:52:51 +01:00

Added subscribers: @Stefan_Werner, @Sergey

blender-admin commented

2020-02-15 09:18:59 +01:00

This issue was referenced by 9531a8a10a

This issue was referenced by 9531a8a10a63ca773ea30d06a960105c89982170

blender-admin commented

2020-02-18 00:04:21 +01:00

This issue was referenced by blender/cycles@1eb0a422a4

This issue was referenced by blender/cycles@1eb0a422a4e05bdc0c50a39ee5579b9c0c800f97

blender-admin commented

2020-02-18 00:04:21 +01:00

This issue was referenced by 1761d65b25

This issue was referenced by 1761d65b25d7161e8597bae9fa0ae7b0cf93c17f

blender-admin commented

2020-02-18 00:20:58 +01:00

This issue was referenced by blender/cycles@12a9031a4a

This issue was referenced by blender/cycles@12a9031a4a2c97535a2ac7f898d2c38607794dff

blender-admin commented

2020-02-18 00:20:58 +01:00

This issue was referenced by ca72ff599a

This issue was referenced by ca72ff599a96b64e6159e8e8f891f9649c0717a3

blender-admin commented

2020-02-18 14:02:40 +01:00

This issue was referenced by blender/cycles@c74107bc7d

This issue was referenced by blender/cycles@c74107bc7d89f4b65f43dda0d6c9ffe6f47f93f0

blender-admin commented

2020-02-18 14:02:40 +01:00

This issue was referenced by cef4d344f9

This issue was referenced by cef4d344f948ce75a7787ad0c3915c87e1716ca1

Brecht Van Lommel commented

2020-02-18 17:20:39 +01:00

It seems Embree does not have an equivalent of our Thick Line Segments. In the current patch they are rendered as Flat/Ribbons, which gives entirely different results for e.g. the principled hair BSDF.

We can either add our own line segment primitive, or hope the Embree curve rendering is fast enough and port it to the GPU.

It seems Embree does not have an equivalent of our Thick Line Segments. In the current patch they are rendered as Flat/Ribbons, which gives entirely different results for e.g. the principled hair BSDF. We can either add our own line segment primitive, or hope the Embree curve rendering is fast enough and port it to the GPU.

Brecht Van Lommel commented

2020-02-18 20:02:40 +01:00

From testing I think all the bad differences are due to the lack of support for thick line segments.

blender-admin commented

2020-02-18 20:54:41 +01:00

This issue was referenced by blender/cycles@828d78ade3

This issue was referenced by blender/cycles@828d78ade3c7c865deb6030e191f3ed11832ad5d

blender-admin commented

2020-02-18 20:54:41 +01:00

This issue was referenced by 207338bb58

This issue was referenced by 207338bb58b1a44c531e6d78fad68672c6d3b2e1

Brecht Van Lommel commented

2020-02-20 13:41:08 +01:00

I ran the basic benchmark scenes with Embree, performance is improved in all cases. Mainly the hair is faster. These scenes don't test motion blur, where the biggest speedups will be.

AMD Ryzen 2990WX, Ubuntu Linux

bmw27	-3.5%
classroom	-0.6%
fishy_cat	-13.1%
koro	-27.1%
pabellon	-13.2%

Peak memory is cut in half in the hair scenes (did not investigate why, but not hard to imagine Embree BVH is better at this), and a couple % lower for other scenes. So in terms of performance and memory usage, everything seems to be fine.

I ran the basic benchmark scenes with Embree, performance is improved in all cases. Mainly the hair is faster. These scenes don't test motion blur, where the biggest speedups will be. **AMD Ryzen 2990WX, Ubuntu Linux** |bmw27|-3.5%| | -- | -- | |classroom|-0.6%| |fishy_cat|-13.1%| |koro|-27.1%| |pabellon|-13.2%| Peak memory is cut in half in the hair scenes (did not investigate why, but not hard to imagine Embree BVH is better at this), and a couple % lower for other scenes. So in terms of performance and memory usage, everything seems to be fine.

Stefan Werner commented

2020-02-20 14:28:26 +01:00

Thick line segments could be emulated through spline primitives at the expense of wasted memory.

Brecht Van Lommel commented

2020-02-20 19:58:14 +01:00

The main purpose of line segments is to have better performance. I'm not sure how expensive spline primitive intersections is compared to curves?

I'm hoping we can just remove line segments if the curve performance is good enough.

The main purpose of line segments is to have better performance. I'm not sure how expensive spline primitive intersections is compared to curves? I'm hoping we can just remove line segments if the curve performance is good enough.

blender-admin commented

2020-02-23 09:14:52 +01:00

This issue was referenced by blender/cycles@a5a82369de

This issue was referenced by blender/cycles@a5a82369de548b718eb7a3f1c2e59ca7388d881a

blender-admin commented

2020-02-23 09:14:52 +01:00

This issue was referenced by 99436acde8

This issue was referenced by 99436acde8fbb7381f095922bb61455b0c8fb9a9

blender-admin commented

2020-02-23 10:15:35 +01:00

This issue was referenced by blender/cycles@7ccace1231

This issue was referenced by blender/cycles@7ccace12313dce218857c7abe37b65c5c8774af7

blender-admin commented

2020-02-23 10:15:35 +01:00

This issue was referenced by 1de0e13af6

This issue was referenced by 1de0e13af619e405f351bf42924f819dc3a9bc44

blender-admin commented

2020-02-24 20:29:01 +01:00

This issue was referenced by blender/cycles@ceeb1e6401

This issue was referenced by blender/cycles@ceeb1e64010cd378ed3c96cc1359c6a93bff068b

blender-admin commented

2020-02-24 20:29:01 +01:00

This issue was referenced by f61f26b033

This issue was referenced by f61f26b033fd7d9c8b8bbf8b77cb24ef902f5ba8

Brecht Van Lommel commented

2020-02-24 21:47:23 +01:00

I've ported the Embree code for rendering catmull-rom curves and ribbons to Cycles for GPU rendering.

Fr thick curves it's significantly slower than our previous code with default number of subdivisions. I haven't worked on optimizing the GPU code (there's a few things to check but not sure there is anything to close the gap). Also on the CPU I'm finding some slower performance now.

It makes sense in that the Embree code subdivides to float error precision, while we only use a fixed number of subdivisions. And it shows in close ups. But if the hair is small, there's not really any need for that kind of precision.

I'll need to check if there are some cheaper things we can do, either reducing the precision for thick curves somehow, or computing normals for ribbons that make them look like thick curves.

embree-curves.patch

I've ported the Embree code for rendering catmull-rom curves and ribbons to Cycles for GPU rendering. Fr thick curves it's significantly slower than our previous code with default number of subdivisions. I haven't worked on optimizing the GPU code (there's a few things to check but not sure there is anything to close the gap). Also on the CPU I'm finding some slower performance now. It makes sense in that the Embree code subdivides to float error precision, while we only use a fixed number of subdivisions. And it shows in close ups. But if the hair is small, there's not really any need for that kind of precision. I'll need to check if there are some cheaper things we can do, either reducing the precision for thick curves somehow, or computing normals for ribbons that make them look like thick curves. [embree-curves.patch](https://archive.blender.org/developer/F8367266/embree-curves.patch)

Brecht Van Lommel commented

2020-03-05 22:54:43 +01:00

So I'm really struggling to get the Embree intersection code for thick curves to perform well on the GPU. It's still around 1.7x slower than our existing curve rendering.

Reducing the number of subdivisions/iterations leads to artifacts and fireflies. I was hoping we could skip backfaces more efficiently by doing it in the intersection code. But I don't see a way to do it with the iterative refinement that the algorithm is using, the far intersection seems to start with a back-facing hit but then can become front-facing as it iterates closer to the solution.

Maybe there are ways to reduce register pressure or other code reorganization that can help, but I'm not seeing obvious ways to do it.

It's also higher quality than our existing code, and just involves a lot more float ops. Probably more than you need in practice unless you're looking at hairs really up close. The next thing I want to try is to use ribbons and compute rounded normals for them. If that works ok it could be a good default, with thick curves as a fallback for when you need to render hair really up close.

For a hair shader like Principled Hair we don't even need a normal.

So I'm really struggling to get the Embree intersection code for thick curves to perform well on the GPU. It's still around 1.7x slower than our existing curve rendering. Reducing the number of subdivisions/iterations leads to artifacts and fireflies. I was hoping we could skip backfaces more efficiently by doing it in the intersection code. But I don't see a way to do it with the iterative refinement that the algorithm is using, the far intersection seems to start with a back-facing hit but then can become front-facing as it iterates closer to the solution. Maybe there are ways to reduce register pressure or other code reorganization that can help, but I'm not seeing obvious ways to do it. It's also higher quality than our existing code, and just involves a lot more float ops. Probably more than you need in practice unless you're looking at hairs really up close. The next thing I want to try is to use ribbons and compute rounded normals for them. If that works ok it could be a good default, with thick curves as a fallback for when you need to render hair really up close. For a hair shader like Principled Hair we don't even need a normal.

Julien Leménicier commented

2020-03-20 17:17:59 +01:00

Added subscriber: @lemenicier_julien