Cycles Procedural API and faster scene update #79174

New Issue

Kévin Dietrich · 2020-07-22T21:57:26+02:00

Kévin Dietrich commented

2020-07-22 21:57:26 +02:00

This design task is for discussing ideas regarding procedural nodes and improvements to the scene update for faster interactive rendering.

Procedural API

A procedural is a node that can create geometry nodes in the scene before rendering. Such nodes can be used to import data from Alembic or USD files, or even a custom file format, without Cycles needing to explicitly know about those formats. The API for such a node can be as simple as deriving a base class and overriding a virtual function as suggested in D3089 or it could be more granular with various functions to know what they would create (e.g. number and types of nodes). Internally, the procedural nodes will use the Cycles API (#79131) to create or update data.

With procedurals the current approach to updating the scene data would need to be changed. Currently, scene updates start by iterating over the geometry nodes and their shaders before calling Scene::device_update(), but procedurals would need to be updated first.

Managing the procedurals, and the nodes that they create is bit unclear at the moment. When working on updating D3089, I run into issues with the Blender scene synchronization stage where Cycles objects not known by the synchronization system would be deleted between frames, or during viewport updates in interactive rendering. I did add some mechanisms to preserve those objects but it is not a nice approach I think.

Faster scene update

Scene update is too slow for real time applications or interactive rendering. Currently, Cycles is using a very conservative approach to updating scene data, especially for mesh nodes: if anything is changed, the entire device memory is freed and recreated. Ideally, if we only changed the vertices of a mesh, without using any displacement, we should be able to just copy the new data directly into the device memory, and recreate the BVH. For mesh objects, the device memory may also be recreated if displacement is used and has been applied, which leads to a double update of the device memory for every frame.

To perform the updates, we iterate over all of the objects and shaders in the scene in a certain order to account for dependencies between them. Because of the dependencies, we may iterate multiple times over the same container. For example, mesh objects are iterated over multiple times during a single update :

once for defining which shaders they used
once for updating their flags
once for generating their data
once for loading the images they use for displacement
once for generating displacement
once for handling volumetric data
once for generating their BVHs

And a few more times to gather metrics to allocate and copy their data to the device. Most of the times though, after the first frame has been rendered, these loops skip over the meshes that do not need an update.

To accelerate data processing, temporary multithreaded task pools are used in various places. For example, there exist dedicated task pools for loading 3d volume textures, images, or compiling SVM shaders.

To improve performance regarding scene updates, we could reduce the number of loops, parallelize object updates, and avoid unnecessary memory reallocations and data copying.

What needs to be known to granularize updates for geometry objects:

if vertices have changed (-> rebuild BVH, recompute normals and displacement if needed)
if vertex count has changed (-> recreate device memory, rebuild BVH, normals, displacement)
if an attribute has changed (-> what is the attribute affecting?)
if an image or a shader affecting displacement has changed
if displacement parameters have changed (-> recompute displacement, BVH, device_memory, normals)
if the triangles shaders changed (-> copy the new data to the memory)
other?

Those flags would be set automatically when modifying data through the API to free clients from setting them directly (#79131).

Ideas for improvements

a data oriented approach

Separate geometries in different arrays based on what needs to be done : geometries_to_displace, geometries_needing_bvh, gometries_needing_volumes, etc. And process arrays separately. This would apply to all the managers: each one would keep a list of managed objects that were modified.

a task system approach

This is the system that best fits a case where procedurals can recursively generate other procedurals.

Scene updates could be handled by a multithreaded task sheduler, replacing the various specific task pools currently used, and making the entire process multithreaded.

Tasks would be initially created for the objects (and the procedurals) which need to be updated. These would create other tasks, if needed, for the shaders that they use, and these in turn would create tasks for loading images, etc. This way, we would remove a lot of the manual iterations over containers to know what is needed. This essentially creates a dependency graph and processes it at the same time.

Tasks can be granular, and some node types can have multiple tasks. A mesh node could have one task for its general data update (computing normals, etc.) one for creating displacement data, and a final one to compute its BVH.

Dependencies between tasks can be noted: a displacement task requiring an image to be processed should wait for the image data to be available. The task scheduler can also segregate tasks by type and hand tasks to worker threads in an order similar to that presently used in Scene::device_update, i.e. image tasks would be processed before displacement tasks.

When all tasks are done, and data about what they did has been gathered, we can then at once decide whether to recreate the device memory or override it with new data and hand it over to the device for rendering.

This design task is for discussing ideas regarding procedural nodes and improvements to the scene update for faster interactive rendering. **Procedural API** A procedural is a node that can create geometry nodes in the scene before rendering. Such nodes can be used to import data from Alembic or USD files, or even a custom file format, without Cycles needing to explicitly know about those formats. The API for such a node can be as simple as deriving a base class and overriding a virtual function as suggested in [D3089](https://archive.blender.org/developer/D3089) or it could be more granular with various functions to know what they would create (e.g. number and types of nodes). Internally, the procedural nodes will use the Cycles API (#79131) to create or update data. With procedurals the current approach to updating the scene data would need to be changed. Currently, scene updates start by iterating over the geometry nodes and their shaders before calling `Scene::device_update()`, but procedurals would need to be updated first. Managing the procedurals, and the nodes that they create is bit unclear at the moment. When working on updating [D3089](https://archive.blender.org/developer/D3089), I run into issues with the Blender scene synchronization stage where Cycles objects not known by the synchronization system would be deleted between frames, or during viewport updates in interactive rendering. I did add some mechanisms to preserve those objects but it is not a nice approach I think. **Faster scene update** Scene update is too slow for real time applications or interactive rendering. Currently, Cycles is using a very conservative approach to updating scene data, especially for mesh nodes: if anything is changed, the entire device memory is freed and recreated. Ideally, if we only changed the vertices of a mesh, without using any displacement, we should be able to just copy the new data directly into the device memory, and recreate the BVH. For mesh objects, the device memory may also be recreated if displacement is used and has been applied, which leads to a double update of the device memory for every frame. To perform the updates, we iterate over all of the objects and shaders in the scene in a certain order to account for dependencies between them. Because of the dependencies, we may iterate multiple times over the same container. For example, mesh objects are iterated over multiple times during a single update : - once for defining which shaders they used - once for updating their flags - once for generating their data - once for loading the images they use for displacement - once for generating displacement - once for handling volumetric data - once for generating their BVHs And a few more times to gather metrics to allocate and copy their data to the device. Most of the times though, after the first frame has been rendered, these loops skip over the meshes that do not need an update. To accelerate data processing, temporary multithreaded task pools are used in various places. For example, there exist dedicated task pools for loading 3d volume textures, images, or compiling SVM shaders. To improve performance regarding scene updates, we could reduce the number of loops, parallelize object updates, and avoid unnecessary memory reallocations and data copying. What needs to be known to granularize updates for geometry objects: - if vertices have changed (-> rebuild BVH, recompute normals and displacement if needed) - if vertex count has changed (-> recreate device memory, rebuild BVH, normals, displacement) - if an attribute has changed (-> what is the attribute affecting?) - if an image or a shader affecting displacement has changed - if displacement parameters have changed (-> recompute displacement, BVH, device_memory, normals) - if the triangles shaders changed (-> copy the new data to the memory) - other? Those flags would be set automatically when modifying data through the API to free clients from setting them directly (#79131). **Ideas for improvements** - a data oriented approach Separate geometries in different arrays based on what needs to be done : geometries_to_displace, geometries_needing_bvh, gometries_needing_volumes, etc. And process arrays separately. This would apply to all the managers: each one would keep a list of managed objects that were modified. - a task system approach This is the system that best fits a case where procedurals can recursively generate other procedurals. Scene updates could be handled by a multithreaded task sheduler, replacing the various specific task pools currently used, and making the entire process multithreaded. Tasks would be initially created for the objects (and the procedurals) which need to be updated. These would create other tasks, if needed, for the shaders that they use, and these in turn would create tasks for loading images, etc. This way, we would remove a lot of the manual iterations over containers to know what is needed. This essentially creates a dependency graph and processes it at the same time. Tasks can be granular, and some node types can have multiple tasks. A mesh node could have one task for its general data update (computing normals, etc.) one for creating displacement data, and a final one to compute its BVH. Dependencies between tasks can be noted: a displacement task requiring an image to be processed should wait for the image data to be available. The task scheduler can also segregate tasks by type and hand tasks to worker threads in an order similar to that presently used in Scene::device_update, i.e. image tasks would be processed before displacement tasks. When all tasks are done, and data about what they did has been gathered, we can then at once decide whether to recreate the device memory or override it with new data and hand it over to the device for rendering.

Kévin Dietrich commented

2020-07-22 21:57:26 +02:00

Added subscribers: @kevindietrich, @fx

Kenzie commented

2020-08-22 11:53:03 +02:00

Added subscriber: @KenzieMac130

Bartosz Moniewski commented

2020-09-29 21:41:45 +02:00

Added subscriber: @BartekMoniewski

blender-admin commented

2020-10-01 23:16:01 +02:00

This issue was referenced by blender/cycles@50fe70268a

This issue was referenced by blender/cycles@50fe70268a0e63f3750e33a22e95345d9201db1b

blender-admin commented

2020-10-01 23:16:01 +02:00

This issue was referenced by edd1164575

This issue was referenced by edd1164575feefda103c73119a98cbd994e53141

blender-admin commented

2020-10-26 21:00:37 +01:00

This issue was referenced by blender/cycles@d4567df044

This issue was referenced by blender/cycles@d4567df044118c9d39125a28845344a8cf4ce88e

blender-admin commented

2020-10-26 21:00:37 +01:00

This issue was referenced by 527f8b32b3

This issue was referenced by 527f8b32b32187f754e5b176db6377736f9cb8ff

Brecht Van Lommel commented

2020-10-28 14:57:23 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

blender-admin commented

2020-11-04 11:17:38 +01:00

This issue was referenced by blender/cycles@588eb94148

This issue was referenced by blender/cycles@588eb94148e3ea62457b666496cbfc3919c44860

blender-admin commented

2020-11-04 11:17:38 +01:00

This issue was referenced by 31a620b942

This issue was referenced by 31a620b9420cab6292b0aa1ea21c9dd1cf70b8bc

Evan Wilson commented

2020-11-21 17:58:08 +01:00

Added subscriber: @EAW

blender-admin commented

2021-01-22 15:01:26 +01:00

This issue was referenced by blender/cycles@0a013f4a0a

This issue was referenced by blender/cycles@0a013f4a0a93206b686461930ebfb82ca40d4764

blender-admin commented

2021-01-22 15:01:26 +01:00

This issue was referenced by bbe6d44928

This issue was referenced by bbe6d44928235cd4a5cfbeaf1a1de78ed861bb92

blender-admin commented

2021-01-25 14:56:57 +01:00

This issue was referenced by 2e67191c86

This issue was referenced by 2e67191c861f2cb148f05af116114e7332b8e789

blender-admin commented

2021-01-25 15:12:00 +01:00

This issue was referenced by b64f0fab06

This issue was referenced by b64f0fab068169a7c379be728aae8994eb893b18

Steffen Dünner commented

2021-02-16 22:27:21 +01:00

Added subscriber: @SteffenD

blender-admin commented

2021-08-19 14:34:01 +02:00

This issue was referenced by 51862c8445

This issue was referenced by 51862c8445c8d5f573d1c86780db9e0972ef14dd

Brecht Van Lommel commented

2021-08-20 16:40:47 +02:00

Added subscriber: @brecht

Brecht Van Lommel commented

2021-08-20 16:40:47 +02:00

Changed status from 'Confirmed' to: 'Resolved'

Brecht Van Lommel closed this issue

2021-08-20 16:40:47 +02:00

Brecht Van Lommel self-assigned this 2021-08-20 16:40:47 +02:00

Brecht Van Lommel commented

2021-08-20 16:40:47 +02:00

We can consider this project finished now, thanks @kevindietrich for the many optimizations. There will be a blog post on code.blender.org later summarizing the results.

Optimizations are of course always happening, and the Alembic procedural may be incrementally improved further, but this particular project is now finished.

We can consider this project finished now, thanks @kevindietrich for the many optimizations. There will be a blog post on code.blender.org later summarizing the results. Optimizations are of course always happening, and the Alembic procedural may be incrementally improved further, but this particular project is now finished.

Thomas Dinges added this to the 2.90 milestone 2023-02-08 16:26:15 +01:00

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Cycles Procedural API and faster scene update #79174