LayerCollection: Invalid operation on main data from within depsgraph-controlled update code. #112534

New Issue

Bastien Montagne · 2023-09-18T17:49:01+02:00

Bastien Montagne commented

2023-09-18 17:49:01 +02:00

System Information
Operating system: N/A
Graphics card: N/A

Blender Version
Broken: Current main
Worked: ???

Short description of error
The move to deferred synchronization of LayerCollections (with underlying Collection hierarchy) seems to have created an unexpected issue with the ViewLayer depsgraph-update code. Essentially, this code should be called from main thread only (for orig data), not worker threads.

The issue is even more critical when a custom depsgraph is used from within a worker thread, which happens e.g. with USD exporter (but also rendering I think).

Exact steps for others to reproduce the error
Here is a backtrace leading to a fatal CLOG and crash. I got it by trying to export in USD a scene from the pets project (040_0050.anim.blend from r3055, but pretty sure this issue will appear randomly with any complex enough file):

./bin/blender() ./source/creator/creator.cc:196: callback_clg_fatal(void*) (0xf3c2740)
./bin/blender() ./intern/clog/clog.c:374: clg_ctx_fatal_action (0x1d201748)
./bin/blender() ./intern/clog/clog.c:546: CLG_logf (0x1d2032e9)
./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1250 (discriminator 4): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2b72)
./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1263 (discriminator 1): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2c5e)
./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1263 (discriminator 1): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2c5e)
./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1392: BKE_layer_collection_sync (0x106b4435)
./bin/blender() ./source/blender/blenkernel/intern/layer.cc:991: BKE_view_layer_synced_ensure (0x106af32c)
./bin/blender() ./source/blender/depsgraph/intern/builder/deg_builder_nodes_view_layer.cc:98: blender::deg::DepsgraphNodeBuilder::build_view_layer(Scene*, ViewLayer*, blender::deg::eDepsNode_LinkedState_Type) (0x14a0401c)
./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline_view_layer.cc:21: blender::deg::ViewLayerBuilderPipeline::build_nodes(blender::deg::DepsgraphNodeBuilder&) (0x14ad7e3d)
./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline.cc:58: blender::deg::AbstractBuilderPipeline::build_step_nodes() (0x14ac886f)
./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline.cc:37: blender::deg::AbstractBuilderPipeline::build() (0x14ac787c)
./bin/blender() ./source/blender/depsgraph/intern/depsgraph_build.cc:235: DEG_graph_build_from_view_layer (0x14b90694)
./bin/blender() ./source/blender/io/usd/intern/usd_capi_export.cc:323: blender::io::usd::export_startjob(void*, bool*, bool*, float*) (0x17a88bf4)
./bin/blender() ./source/blender/windowmanager/intern/wm_jobs.cc:388: do_job_thread(void*) (0x13c5fcf9)
./bin/blender() ./source/blender/blenlib/intern/threads.cc:178: tslot_thread_start(void*) (0x13b4025a)

Am not entirely sure what is the best approach here (assuming own analysis of the issue above is correct). I would not expect though that depsgraph building (or evaluating) code has to ensure its orig data is valid and up-to-date? So we likely need some function to ensure that this kind of deferred update of orig data is done before starting depsgraph building?

**System Information** Operating system: N/A Graphics card: N/A **Blender Version** Broken: Current main Worked: ??? **Short description of error** The move to deferred synchronization of LayerCollections (with underlying Collection hierarchy) seems to have created an unexpected issue with the ViewLayer depsgraph-update code. Essentially, this code should be called from main thread only (for orig data), not worker threads. The issue is even more critical when a custom depsgraph is used from within a worker thread, which happens e.g. with USD exporter (but also rendering I think). **Exact steps for others to reproduce the error** Here is a backtrace leading to a fatal CLOG and crash. I got it by trying to export in USD a scene from the pets project (`040_0050.anim.blend` from `r3055`, but pretty sure this issue will appear randomly with any complex enough file): ``` ./bin/blender() ./source/creator/creator.cc:196: callback_clg_fatal(void*) (0xf3c2740) ./bin/blender() ./intern/clog/clog.c:374: clg_ctx_fatal_action (0x1d201748) ./bin/blender() ./intern/clog/clog.c:546: CLG_logf (0x1d2032e9) ./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1250 (discriminator 4): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2b72) ./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1263 (discriminator 1): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2c5e) ./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1263 (discriminator 1): view_layer_objects_base_cache_validate(ViewLayer*, LayerCollection*) (0x106b2c5e) ./bin/blender() ./source/blender/blenkernel/intern/layer.cc:1392: BKE_layer_collection_sync (0x106b4435) ./bin/blender() ./source/blender/blenkernel/intern/layer.cc:991: BKE_view_layer_synced_ensure (0x106af32c) ./bin/blender() ./source/blender/depsgraph/intern/builder/deg_builder_nodes_view_layer.cc:98: blender::deg::DepsgraphNodeBuilder::build_view_layer(Scene*, ViewLayer*, blender::deg::eDepsNode_LinkedState_Type) (0x14a0401c) ./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline_view_layer.cc:21: blender::deg::ViewLayerBuilderPipeline::build_nodes(blender::deg::DepsgraphNodeBuilder&) (0x14ad7e3d) ./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline.cc:58: blender::deg::AbstractBuilderPipeline::build_step_nodes() (0x14ac886f) ./bin/blender() ./source/blender/depsgraph/intern/builder/pipeline.cc:37: blender::deg::AbstractBuilderPipeline::build() (0x14ac787c) ./bin/blender() ./source/blender/depsgraph/intern/depsgraph_build.cc:235: DEG_graph_build_from_view_layer (0x14b90694) ./bin/blender() ./source/blender/io/usd/intern/usd_capi_export.cc:323: blender::io::usd::export_startjob(void*, bool*, bool*, float*) (0x17a88bf4) ./bin/blender() ./source/blender/windowmanager/intern/wm_jobs.cc:388: do_job_thread(void*) (0x13c5fcf9) ./bin/blender() ./source/blender/blenlib/intern/threads.cc:178: tslot_thread_start(void*) (0x13b4025a) ``` ------------------------ Am not entirely sure what is the best approach here (assuming own analysis of the issue above is correct). I would not expect though that depsgraph building (or evaluating) code has to ensure its orig data is valid and up-to-date? So we likely need some function to ensure that this kind of deferred update of orig data is done before starting depsgraph building?

Bastien Montagne added the

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

LayerCollection: Invalid operation on main data from within depsgraph-controlled update code. #112534