Fix ViewLayers cache building #73411

New Issue

Bastien Montagne · 2020-01-26T17:50:23+01:00

Bastien Montagne commented

2020-01-26 17:50:23 +01:00

Currently, ViewLayers' collection and objects cache is rebuilt way too much (it can be thousands of time during a single edit action in a relatively heavy scene).

Proposal

Use a lazy update approach:

Code changing the the data only tags the relevant view layers' caches to be updated.
Code actually using those caches is responsible to call a new 'ensure()' callback first, which will check those flags and rebuild the cache as needed.

We can stay fairly general (tag a whole cache as dirty, and just fully rebuild it then when required), or be more specific by tagging every ID that changed, and then cache rebuilding code could only update the parts that need it.

Profiling on worst-case scenarii should tell us if high level of granularity is really needed here (worst case would be a viewlayer with many collections and objects).

Another possibility would be to add an API to actually update the needed parts of the caches on the fly (called from code modifying the related IDs relationships), but this is likely more complex to get working.

Note: Thread safety should not be an issue here, at least there is none at all in current code…

Currently, ViewLayers' collection and objects cache is rebuilt way too much (it can be thousands of time during a single edit action in a relatively heavy scene). ## Proposal Use a lazy update approach: * Code changing the the data only tags the relevant view layers' caches to be updated. * Code actually using those caches is responsible to call a new 'ensure()' callback first, which will check those flags and rebuild the cache as needed. We can stay fairly general (tag a whole cache as dirty, and just fully rebuild it then when required), or be more specific by tagging every ID that changed, and then cache rebuilding code could only update the parts that need it. Profiling on worst-case scenarii should tell us if high level of granularity is really needed here (worst case would be a viewlayer with many collections and objects). Another possibility would be to add an API to actually update the needed parts of the caches on the fly (called from code modifying the related IDs relationships), but this is likely more complex to get working. *Note: Thread safety should not be an issue here, at least there is none at all in current code…*

Bastien Montagne commented

2020-01-26 17:50:23 +01:00

Changed status from 'Needs Triage' to: 'Confirmed'

Bastien Montagne commented

2020-01-26 17:50:23 +01:00

Added subscriber: @mont29

Roman commented

2020-01-26 18:14:03 +01:00

Added subscriber: @rwman

Sam Green commented

2020-02-26 10:45:07 +01:00

Added subscriber: @SamGreen

Dion Moult commented

2020-03-06 06:22:14 +01:00

Added subscriber: @Moult

Ryan Inch commented

2021-09-23 08:39:40 +02:00

Added subscriber: @Imaginer

Daniel Gryningstjerna commented

2022-07-06 17:21:44 +02:00

Added subscriber: @Dangry

Ahmed Hindy commented

2022-07-26 20:31:31 +02:00

Added subscriber: @ahmed.hindy96

Jeroen Bakker commented

2022-08-17 16:59:07 +02:00

Added subscriber: @Jeroen-Bakker

Monique Dewanchand commented

2022-08-17 17:03:18 +02:00

Added subscriber: @monique

Aras Pranckevicius commented

2022-08-18 13:08:23 +02:00

Added subscriber: @aras_p

Aras Pranckevicius commented

2022-08-18 13:08:23 +02:00

For people who aren't super familiar with blender codebase but might want to try doing this (asking for a friend :)) -- is the idea that all the work currently done by the likes of BKE_main_collection_sync, BKE_scene_collection_sync, BKE_layer_collection_sync and similar would merely mark something as "yo, this needs a sync at some point", and then the actual resync happens once it's needed (e.g. when BKE_view_layer_base_find is called)?

And then part of the work would be finding various loops in the codebase that right now do something like

for N objects:
    do something that changes view layer things
    immediately find view layer base for said object

and change them to a pattern like:

for N objects:
    do something that changes view layer things
for N objects:
   find view layer base for the object

since that would turn N heavy operations into 1 heavy operation?

Or am I misunderstanding the issue completely :)

For people who aren't super familiar with blender codebase but might want to try doing this (asking for a friend :)) -- is the idea that all the work currently done by the likes of `BKE_main_collection_sync`, `BKE_scene_collection_sync`, `BKE_layer_collection_sync` and similar would merely mark something as "yo, this needs a sync at some point", and then the actual resync happens once it's needed (e.g. when `BKE_view_layer_base_find` is called)? And then part of the work would be finding various loops in the codebase that right now do something like ``` for N objects: do something that changes view layer things immediately find view layer base for said object ``` and change them to a pattern like: ``` for N objects: do something that changes view layer things for N objects: find view layer base for the object ``` since that would turn N heavy operations into 1 heavy operation? Or am I misunderstanding the issue completely :)

Bastien Montagne commented

2022-08-18 14:25:30 +02:00

@aras_p that is the idea yes. Most (if not all) of the code in BKE_main_collection_sync and similar would then move to the new 'lazy' on-demand syncing code (the ensure() callback in task above).

@aras_p that is the idea yes. Most (if not all) of the code in `BKE_main_collection_sync` and similar would then move to the new 'lazy' on-demand syncing code (the `ensure()` callback in task above).

Monique Dewanchand self-assigned this 2022-08-27 23:09:28 +02:00

blender-admin commented

2022-09-01 10:00:53 +02:00

This issue was referenced by 17501c146e

This issue was referenced by 17501c146edc4af8a5e04565dc4d0b30ed5c5323

Alexey Adamitsky commented

2022-09-14 14:53:01 +02:00

Added subscriber: @AlexeyAdamitsky

blender-admin commented

2022-09-14 21:30:20 +02:00

This issue was referenced by 23276bcc37

This issue was referenced by 23276bcc37acc54f1e1814abdf482a432523c3a6

blender-admin commented

2022-09-14 21:33:51 +02:00

This issue was referenced by 68589a31eb

This issue was referenced by 68589a31ebfb79165f99a979357d237e5413e904

blender-admin commented

2022-10-20 16:14:03 +02:00

This issue was referenced by f017fdecef

This issue was referenced by f017fdecefc41ffe796ccfe9a1cb05772a7ad315

Philipp Oeser removed the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Fix ViewLayers cache building #73411

Proposal