Bad data editing by during DEG evaluation? #53800

New Issue

Bastien Montagne · 2018-01-16T12:51:54+01:00

Bastien Montagne commented

2018-01-16 12:51:54 +01:00

Short description of error

Currently, creating a new static override from linked object in Blender (using spacebar-menu -> 'make_override') crashes in undo step creation code, due to use-after-free memory access.

Exact steps for others to reproduce the error

Use a Debug build of Blender with ASAN enabled.
From default startup, link an object.
Select that linked object.
Hit space, type in 'over' and execute make_override operator.
If it does not crash immediately, trying e.g. to select something else in the scene should trigger to bug.

Detailed ASAN trace:

=================================================================
==4409==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d00086e3e8 at pc 0x5636e9bd9438 bp 0x7ffdebbbfea0 sp 0x7ffdebbbfe98
READ of size 8 at 0x60d00086e3e8 thread T0
    - 0 0x5636e9bd9437 in rna_iterator_listbase_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:4126
    - 1 0x5636e9ec9548 in rna_Object_collection_properties_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_object.c:249
    - 2 0x5636e9edc65b in Object_collection_properties_begin /home/i74700deb64/blender/__work__/build_cmake_dbg/source/blender/makesrna/intern/rna_object_gen.c:1759
    - 3 0x5636e9bd2f74 in RNA_property_collection_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:3335
    - 4 0x5636e9f579b0 in rna_property_override_diff_default /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_rna.c:1435
    - 5 0x5636e9beaec2 in rna_property_override_diff /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:7198
    - 6 0x5636e9bec2d2 in RNA_struct_override_matches /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:7415
    - 7 0x5636e96dc94e in BKE_override_static_operations_create /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/library_override.c:526
    - 8 0x5636e96dcd24 in BKE_main_override_static_operations_create /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/library_override.c:562
    - 9 0x5636e762659e in ED_undo_push /home/i74700deb64/blender/__work__/src/source/blender/editors/util/undo.c:87
    - 10 0x5636e7627157 in ED_undo_push_op /home/i74700deb64/blender/__work__/src/source/blender/editors/util/undo.c:250
    - 11 0x5636e700b197 in wm_operator_finished /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:830
    - 12 0x5636e700e13b in wm_operator_invoke /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:1244
    - 13 0x5636e7011a17 in wm_handler_operator_call /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:1907
    - 14 0x5636e70133a1 in wm_handlers_do_intern /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2187
    - 15 0x5636e701487a in wm_handlers_do /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2448
    - 16 0x5636e7016bee in wm_event_do_handlers /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2836
    - 17 0x5636e6ff4e56 in WM_main /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm.c:521
    - 18 0x5636e6fea6b9 in main /home/i74700deb64/blender/__work__/src/source/creator/creator.c:527
    - 19 0x7f4497e46f29 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20f29)
    #20 0x5636e6fe9979 in _start (/home/i74700deb64/blender/__work__/build_cmake_dbg/bin/blender+0x37b9979)

0x60d00086e3e8 is located 104 bytes inside of 136-byte region [0x60d00086e380,0x60d00086e408)
freed by thread T14 here:
    - 0 0x7f44a3e4b8c8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xd98c8)
    - 1 0x5636ea8806d7 in MEM_lockfree_freeN /home/i74700deb64/blender/__work__/src/intern/guardedalloc/intern/mallocn_lockfree_impl.c:164
    - 2 0x5636e993ea1b in idproperty_reset /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2068
    - 3 0x5636e993ec93 in BKE_layer_eval_layer_collection_pre /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2085
    - 4 0x5636ea357089 in void std::__invoke_impl<void, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(std::__invoke_other, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:60
    - 5 0x5636ea356b44 in std::__invoke_result<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>::type std::__invoke<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:95
    - 6 0x5636ea3564ac in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::__call<void, EvaluationContext*&&, 0ul, 1ul, 2ul>(std::tuple<EvaluationContext*&&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/functional:467
    - 7 0x5636ea355a43 in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::operator()<EvaluationContext*, void>(EvaluationContext*&&) /usr/include/c++/7/functional:551
    - 8 0x5636ea354cdf in std::_Function_handler<void (EvaluationContext*), std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)> >::_M_invoke(std::_Any_data const&, EvaluationContext*&&) /usr/include/c++/7/bits/std_function.h:316
    - 9 0x5636ea33e4fe in std::function<void (EvaluationContext*)>::operator()(EvaluationContext*) const /usr/include/c++/7/bits/std_function.h:706
    - 10 0x5636ea33cc74 in deg_task_run_func /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:93
    - 11 0x5636ea2bea7c in task_scheduler_thread_run /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:441
    #12 0x7f44a1f7f519 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7519)

previously allocated by thread T0 here:
    - 0 0x7f44a3e4bdf8 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xd9df8)
    - 1 0x5636ea880bdd in MEM_lockfree_callocN /home/i74700deb64/blender/__work__/src/intern/guardedalloc/intern/mallocn_lockfree_impl.c:281
    - 2 0x5636e96793fb in IDP_New /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/idprop.c:1033
    - 3 0x5636e993ea33 in idproperty_reset /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2070
    - 4 0x5636e993ec93 in BKE_layer_eval_layer_collection_pre /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2085
    - 5 0x5636ea357089 in void std::__invoke_impl<void, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(std::__invoke_other, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:60
    - 6 0x5636ea356b44 in std::__invoke_result<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>::type std::__invoke<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:95
    - 7 0x5636ea3564ac in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::__call<void, EvaluationContext*&&, 0ul, 1ul, 2ul>(std::tuple<EvaluationContext*&&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/functional:467
    - 8 0x5636ea355a43 in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::operator()<EvaluationContext*, void>(EvaluationContext*&&) /usr/include/c++/7/functional:551
    - 9 0x5636ea354cdf in std::_Function_handler<void (EvaluationContext*), std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)> >::_M_invoke(std::_Any_data const&, EvaluationContext*&&) /usr/include/c++/7/bits/std_function.h:316
    - 10 0x5636ea33e4fe in std::function<void (EvaluationContext*)>::operator()(EvaluationContext*) const /usr/include/c++/7/bits/std_function.h:706
    - 11 0x5636ea33cc74 in deg_task_run_func /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:93
    - 12 0x5636ea2c24b5 in BLI_task_pool_work_and_wait /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:893
    - 13 0x5636ea33e379 in DEG::deg_evaluate_on_refresh(EvaluationContext*, DEG::Depsgraph*) /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:263
    - 14 0x5636ea2ef491 in DEG_evaluate_on_refresh /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/depsgraph_eval.cc:106
    - 15 0x5636e9962b79 in BKE_scene_graph_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/scene.c:1434
    - 16 0x5636e9ac3dd1 in BKE_workspace_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/workspace.c:522
    - 17 0x5636e7007fab in wm_event_do_refresh_wm_and_depsgraph /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:342
    - 18 0x5636e7008f93 in wm_event_do_notifiers /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:495
    - 19 0x5636e6ff4e62 in WM_main /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm.c:524
    - 20 0x5636e6fea6b9 in main /home/i74700deb64/blender/__work__/src/source/creator/creator.c:527
    - 21 0x7f4497e46f29 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20f29)

Thread T14 created by T0 here:
    - 0 0x7f44a3da9390 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37390)
    - 1 0x5636ea2bf3e4 in BLI_task_scheduler_create /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:504
    - 2 0x5636ea2c669e in BLI_task_scheduler_get /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/threads.c:176
    - 3 0x5636ea2c50f6 in BLI_task_parallel_range /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:1099
    - 4 0x5636ea342f8c in flush_prepare /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval_flush.cc:104
    - 5 0x5636ea342f8c in DEG::deg_graph_flush_updates(Main*, DEG::Depsgraph*) /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval_flush.cc:264
    - 6 0x5636ea2f669e in DEG_graph_flush_update /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/depsgraph_tag.cc:428
    - 7 0x5636e9962b66 in BKE_scene_graph_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/scene.c:1430
    - 8 0x5636e9f693f6 in rna_Scene_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_scene_api.c:143
    - 9 0x5636e9f9c11a in Scene_update_call /home/i74700deb64/blender/__work__/build_cmake_dbg/source/blender/makesrna/intern/rna_scene_gen.c:8873
    - 10 0x5636e9be63ec in RNA_function_call /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:6475
    - 11 0x5636e83d537a in pyrna_func_call /home/i74700deb64/blender/__work__/src/source/blender/python/intern/bpy_rna.c:5682
    #12 0x7f44a302a218 in _PyObject_FastCallDict (/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0+0x7c218)

So looks like RNA diffing code (called from 'undo_step' function) is accessing some Object's data (ID props), while that same data is being replaced in threaded scene update.

I might be wrong here, but to me allowing deg evaluation to mess with 'real' data is pure evil, afaik we are trying to avoid that completely with CoW? But still, would be better to avoid that kind of behavior, this can also crash randomly when accessing data from py script (including UI) e.g.

Worth noting that with CoW enabled, it does not crash, btw.

**Short description of error** Currently, creating a new static override from linked object in Blender (using spacebar-menu -> 'make_override') crashes in undo step creation code, due to use-after-free memory access. **Exact steps for others to reproduce the error** * Use a Debug build of Blender with ASAN enabled. * From default startup, link an object. * Select that linked object. * Hit space, type in 'over' and execute make_override operator. * If it does not crash immediately, trying e.g. to select something else in the scene should trigger to bug. Detailed ASAN trace: ``` ================================================================= ==4409==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d00086e3e8 at pc 0x5636e9bd9438 bp 0x7ffdebbbfea0 sp 0x7ffdebbbfe98 READ of size 8 at 0x60d00086e3e8 thread T0 - 0 0x5636e9bd9437 in rna_iterator_listbase_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:4126 - 1 0x5636e9ec9548 in rna_Object_collection_properties_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_object.c:249 - 2 0x5636e9edc65b in Object_collection_properties_begin /home/i74700deb64/blender/__work__/build_cmake_dbg/source/blender/makesrna/intern/rna_object_gen.c:1759 - 3 0x5636e9bd2f74 in RNA_property_collection_begin /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:3335 - 4 0x5636e9f579b0 in rna_property_override_diff_default /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_rna.c:1435 - 5 0x5636e9beaec2 in rna_property_override_diff /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:7198 - 6 0x5636e9bec2d2 in RNA_struct_override_matches /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:7415 - 7 0x5636e96dc94e in BKE_override_static_operations_create /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/library_override.c:526 - 8 0x5636e96dcd24 in BKE_main_override_static_operations_create /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/library_override.c:562 - 9 0x5636e762659e in ED_undo_push /home/i74700deb64/blender/__work__/src/source/blender/editors/util/undo.c:87 - 10 0x5636e7627157 in ED_undo_push_op /home/i74700deb64/blender/__work__/src/source/blender/editors/util/undo.c:250 - 11 0x5636e700b197 in wm_operator_finished /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:830 - 12 0x5636e700e13b in wm_operator_invoke /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:1244 - 13 0x5636e7011a17 in wm_handler_operator_call /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:1907 - 14 0x5636e70133a1 in wm_handlers_do_intern /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2187 - 15 0x5636e701487a in wm_handlers_do /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2448 - 16 0x5636e7016bee in wm_event_do_handlers /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:2836 - 17 0x5636e6ff4e56 in WM_main /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm.c:521 - 18 0x5636e6fea6b9 in main /home/i74700deb64/blender/__work__/src/source/creator/creator.c:527 - 19 0x7f4497e46f29 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20f29) #20 0x5636e6fe9979 in _start (/home/i74700deb64/blender/__work__/build_cmake_dbg/bin/blender+0x37b9979) 0x60d00086e3e8 is located 104 bytes inside of 136-byte region [0x60d00086e380,0x60d00086e408) freed by thread T14 here: - 0 0x7f44a3e4b8c8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xd98c8) - 1 0x5636ea8806d7 in MEM_lockfree_freeN /home/i74700deb64/blender/__work__/src/intern/guardedalloc/intern/mallocn_lockfree_impl.c:164 - 2 0x5636e993ea1b in idproperty_reset /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2068 - 3 0x5636e993ec93 in BKE_layer_eval_layer_collection_pre /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2085 - 4 0x5636ea357089 in void std::__invoke_impl<void, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(std::__invoke_other, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:60 - 5 0x5636ea356b44 in std::__invoke_result<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>::type std::__invoke<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:95 - 6 0x5636ea3564ac in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::__call<void, EvaluationContext*&&, 0ul, 1ul, 2ul>(std::tuple<EvaluationContext*&&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/functional:467 - 7 0x5636ea355a43 in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::operator()<EvaluationContext*, void>(EvaluationContext*&&) /usr/include/c++/7/functional:551 - 8 0x5636ea354cdf in std::_Function_handler<void (EvaluationContext*), std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)> >::_M_invoke(std::_Any_data const&, EvaluationContext*&&) /usr/include/c++/7/bits/std_function.h:316 - 9 0x5636ea33e4fe in std::function<void (EvaluationContext*)>::operator()(EvaluationContext*) const /usr/include/c++/7/bits/std_function.h:706 - 10 0x5636ea33cc74 in deg_task_run_func /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:93 - 11 0x5636ea2bea7c in task_scheduler_thread_run /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:441 #12 0x7f44a1f7f519 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x7519) previously allocated by thread T0 here: - 0 0x7f44a3e4bdf8 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xd9df8) - 1 0x5636ea880bdd in MEM_lockfree_callocN /home/i74700deb64/blender/__work__/src/intern/guardedalloc/intern/mallocn_lockfree_impl.c:281 - 2 0x5636e96793fb in IDP_New /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/idprop.c:1033 - 3 0x5636e993ea33 in idproperty_reset /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2070 - 4 0x5636e993ec93 in BKE_layer_eval_layer_collection_pre /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/layer.c:2085 - 5 0x5636ea357089 in void std::__invoke_impl<void, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(std::__invoke_other, void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:60 - 6 0x5636ea356b44 in std::__invoke_result<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>::type std::__invoke<void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*, ID*&, ViewLayer*&>(void (*&)(EvaluationContext const*, ID*, ViewLayer*), EvaluationContext*&&, ID*&, ViewLayer*&) /usr/include/c++/7/bits/invoke.h:95 - 7 0x5636ea3564ac in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::__call<void, EvaluationContext*&&, 0ul, 1ul, 2ul>(std::tuple<EvaluationContext*&&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/include/c++/7/functional:467 - 8 0x5636ea355a43 in void std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)>::operator()<EvaluationContext*, void>(EvaluationContext*&&) /usr/include/c++/7/functional:551 - 9 0x5636ea354cdf in std::_Function_handler<void (EvaluationContext*), std::_Bind<void (*(std::_Placeholder<1>, ID*, ViewLayer*))(EvaluationContext const*, ID*, ViewLayer*)> >::_M_invoke(std::_Any_data const&, EvaluationContext*&&) /usr/include/c++/7/bits/std_function.h:316 - 10 0x5636ea33e4fe in std::function<void (EvaluationContext*)>::operator()(EvaluationContext*) const /usr/include/c++/7/bits/std_function.h:706 - 11 0x5636ea33cc74 in deg_task_run_func /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:93 - 12 0x5636ea2c24b5 in BLI_task_pool_work_and_wait /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:893 - 13 0x5636ea33e379 in DEG::deg_evaluate_on_refresh(EvaluationContext*, DEG::Depsgraph*) /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval.cc:263 - 14 0x5636ea2ef491 in DEG_evaluate_on_refresh /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/depsgraph_eval.cc:106 - 15 0x5636e9962b79 in BKE_scene_graph_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/scene.c:1434 - 16 0x5636e9ac3dd1 in BKE_workspace_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/workspace.c:522 - 17 0x5636e7007fab in wm_event_do_refresh_wm_and_depsgraph /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:342 - 18 0x5636e7008f93 in wm_event_do_notifiers /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm_event_system.c:495 - 19 0x5636e6ff4e62 in WM_main /home/i74700deb64/blender/__work__/src/source/blender/windowmanager/intern/wm.c:524 - 20 0x5636e6fea6b9 in main /home/i74700deb64/blender/__work__/src/source/creator/creator.c:527 - 21 0x7f4497e46f29 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x20f29) Thread T14 created by T0 here: - 0 0x7f44a3da9390 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37390) - 1 0x5636ea2bf3e4 in BLI_task_scheduler_create /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:504 - 2 0x5636ea2c669e in BLI_task_scheduler_get /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/threads.c:176 - 3 0x5636ea2c50f6 in BLI_task_parallel_range /home/i74700deb64/blender/__work__/src/source/blender/blenlib/intern/task.c:1099 - 4 0x5636ea342f8c in flush_prepare /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval_flush.cc:104 - 5 0x5636ea342f8c in DEG::deg_graph_flush_updates(Main*, DEG::Depsgraph*) /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/eval/deg_eval_flush.cc:264 - 6 0x5636ea2f669e in DEG_graph_flush_update /home/i74700deb64/blender/__work__/src/source/blender/depsgraph/intern/depsgraph_tag.cc:428 - 7 0x5636e9962b66 in BKE_scene_graph_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/blenkernel/intern/scene.c:1430 - 8 0x5636e9f693f6 in rna_Scene_update_tagged /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_scene_api.c:143 - 9 0x5636e9f9c11a in Scene_update_call /home/i74700deb64/blender/__work__/build_cmake_dbg/source/blender/makesrna/intern/rna_scene_gen.c:8873 - 10 0x5636e9be63ec in RNA_function_call /home/i74700deb64/blender/__work__/src/source/blender/makesrna/intern/rna_access.c:6475 - 11 0x5636e83d537a in pyrna_func_call /home/i74700deb64/blender/__work__/src/source/blender/python/intern/bpy_rna.c:5682 #12 0x7f44a302a218 in _PyObject_FastCallDict (/usr/lib/x86_64-linux-gnu/libpython3.6m.so.1.0+0x7c218) ``` So looks like RNA diffing code (called from 'undo_step' function) is accessing some Object's data (ID props), while that same data is being replaced in threaded scene update. I might be wrong here, but to me allowing deg evaluation to mess with 'real' data is pure evil, afaik we are trying to avoid that completely with CoW? But still, would be better to avoid that kind of behavior, this can also crash randomly when accessing data from py script (including UI) e.g. Worth noting that with CoW enabled, it does not crash, btw.

Bastien Montagne commented

2018-01-16 12:51:54 +01:00

Added subscriber: @mont29

Bastien Montagne commented

2018-01-16 12:52:17 +01:00

Added subscriber: @dfelinto

Dalai Felinto was assigned by Bastien Montagne

2018-01-16 14:02:31 +01:00

Dalai Felinto commented

2018-03-19 22:23:40 +01:00

Good news, I managed to reproduce this in a unit test: P638. I'm still not sure of what is going on though.

Good news, I managed to reproduce this in a unit test: [P638](https://archive.blender.org/developer/P638.txt). I'm still not sure of what is going on though.

Dalai Felinto commented

2018-03-20 21:23:27 +01:00

Technically we can remove this (ob.collection_properties) from the RNA. I added it there mostly because of the unittests - this way I could see if the dynamic override system was working.

I would rather not remove it though. Can't we flag some RNA properties to be skipped from RNA diff? Or is this the only real-time data that is exposed via RNA (and written in threads)?

That won't change the fact that we are indeed creating run-time evaluation data in depsgraph. But I don't see the issue of this. It's not different than setting object final visibility via depsgraph, stored in its ob->base_flag.

Technically we can remove this (`ob.collection_properties`) from the RNA. I added it there mostly because of the unittests - this way I could see if the dynamic override system was working. I would rather not remove it though. Can't we flag some RNA properties to be skipped from RNA diff? Or is this the only real-time data that is exposed via RNA (and written in threads)? That won't change the fact that we are indeed creating run-time evaluation data in depsgraph. But I don't see the issue of this. It's not different than setting object final visibility via depsgraph, stored in its `ob->base_flag`.

Sergey Sharybin commented

2018-03-21 17:37:09 +01:00

Added subscriber: @Sergey

Sergey Sharybin commented

2018-03-21 17:37:09 +01:00

This is either due to missed update flush to the object (which could be fixed from dependency graph side). But it is also very well possible is that flush is not needed on that object, and that you're just referencing freed data. Similar to what could happen with other evaluated data, like derived mesh.

On the one hand, you should skip evaluated data from diff. But on another hand, you can not skip object's data which will both be holding original datablock (for data from bmain) or evaluated datablock (for objects from depsgraph).

I would also think that make override should operate on the original datablock, which will not reference anything from it's collection properties. So, making CoW enabled by default (and not allowing to disable it) would solve the issue.

This is either due to missed update flush to the object (which could be fixed from dependency graph side). But it is also very well possible is that flush is not needed on that object, and that you're just referencing freed data. Similar to what could happen with other evaluated data, like derived mesh. On the one hand, you should skip evaluated data from diff. But on another hand, you can not skip object's data which will both be holding original datablock (for data from bmain) or evaluated datablock (for objects from depsgraph). I would also think that make override should operate on the original datablock, which will not reference anything from it's collection properties. So, making CoW enabled by default (and not allowing to disable it) would solve the issue.

Dalai Felinto removed their assignment 2018-03-23 23:57:16 +01:00

Bastien Montagne was assigned by Dalai Felinto

2018-03-23 23:57:16 +01:00

Dalai Felinto commented

2018-03-23 23:57:16 +01:00

As discussed over IRC with Bastien, this can be fixed by ignoring real-time evaluated properties for the RNA diff 0c753c1dc4.

As discussed over IRC with Bastien, this can be fixed by ignoring real-time evaluated properties for the RNA diff 0c753c1dc4a.

Bastien Montagne commented

2018-03-26 10:38:49 +02:00

@dfelinto I will do that yes (though I could not reproduce crash with current blender2.8 anymore… go figure).

Point remains, that imho it should not be allowed for threaded depsgraph to directly affect data, not when something else might be running in main thread. Am a bit wondering how it is possible actually for main thread to be doing undo step, while depsgraph is running in some other threads? Is new deg using a deferred/background? I would expect DEG to call 'process tasks and wait' from main thread…

@dfelinto I will do that yes (though I could not reproduce crash with current blender2.8 anymore… go figure). Point remains, that imho it should not be allowed for threaded depsgraph to directly affect data, not when something else might be running in main thread. Am a bit wondering how it is possible actually for main thread to be doing undo step, while depsgraph is running in some other threads? Is new deg using a deferred/background? I would expect DEG to call 'process tasks and wait' from main thread…

blender-admin commented

2018-03-26 10:41:31 +02:00

This issue was referenced by 13f80a152a

This issue was referenced by 13f80a152a3a84286330b1bc6d9818279b976e91

Bastien Montagne commented

2018-03-26 10:43:25 +02:00

Changed status from 'Open' to: 'Resolved'

Bastien Montagne closed this issue

2018-03-26 10:43:25 +02:00

Sergey Sharybin commented

2018-03-26 10:55:25 +02:00

@mont29, the only case when depsgraph might change some of the data is for things like material review. But those don't use objects from main bmain, only materials are shared and no write to them is happening to my knowledge. Undo step is supposed to stop all the running jobs.

Depsgraph does not do any deferred tasks, all its threads are done after DEG_evaluate_on_refresh() and DEG_evaluate_on_framechange() are done.

Possible reasons why depsgraph threads are active during undo:

Preview job is still running (which is not supposed to happen).
Memory compression which is sort of deferred thread is running (though, not sure it will do anything with the depsgraph).

Are you sure there are depsgraph threads active during undo? To me it seems more like due to changes in relations, some of the runtime only-data is not evaluated/assigned by dependency graph (since object is no longer in the graph or so), but is being accessed by generic RNA traversal algorithm (which dereferences previously assigned pointer).

@mont29, the only case when depsgraph might change some of the data is for things like material review. But those don't use objects from main bmain, only materials are shared and no write to them is happening to my knowledge. Undo step is supposed to stop all the running jobs. Depsgraph does not do any deferred tasks, all its threads are done after `DEG_evaluate_on_refresh()` and `DEG_evaluate_on_framechange()` are done. Possible reasons why depsgraph threads are active during undo: - Preview job is still running (which is not supposed to happen). - Memory compression which is sort of deferred thread is running (though, not sure it will do anything with the depsgraph). Are you sure there are depsgraph threads active during undo? To me it seems more like due to changes in relations, some of the runtime only-data is not evaluated/assigned by dependency graph (since object is no longer in the graph or so), but is being accessed by generic RNA traversal algorithm (which dereferences previously assigned pointer).

Bastien Montagne commented

2018-03-26 14:05:54 +02:00

@Sergey It’s hard to be sure of anything, especially since I cannot reproduce the issue anymore currently. But, reading ASAN report from initial post:

Thread0: RNA accessor of Object.collection_properties merely accesses (DNA) Object.base_collection_properties, which is not NULL but points to freed memory
Thread14: That memory has been freed by some DEG-called layer process, which ends up calling idproperty_reset(), and that func immediately re-assign object->base_collection_properties with newly allocated mem.

I’m not sure to see how this access-after-free could happen, besides a concurrent situation between execution of RNA accessor (from undo code) and idproperty_reset (from DEG)?

Note that thread14 is created from DEG_graph_flush_update(), triggered from RNA's Scene.update() (py API), not regular DEG process I believe? At least, it does not seem to go through DEG_evaluate_on_refresh() nor DEG_evaluate_on_framechange()…

@Sergey It’s hard to be sure of anything, especially since I cannot reproduce the issue anymore currently. But, reading ASAN report from initial post: * Thread0: RNA accessor of Object.collection_properties merely accesses (DNA) Object.base_collection_properties, which is not NULL but points to freed memory * Thread14: That memory has been freed by some DEG-called layer process, which ends up calling `idproperty_reset()`, and that func immediately re-assign object->base_collection_properties with newly allocated mem. I’m not sure to see how this access-after-free could happen, besides a concurrent situation between execution of RNA accessor (from undo code) and idproperty_reset (from DEG)? Note that thread14 is created from `DEG_graph_flush_update()`, triggered from RNA's Scene.update() (py API), not regular DEG process I believe? At least, it does not seem to go through `DEG_evaluate_on_refresh()` nor `DEG_evaluate_on_framechange()`…

Sergey Sharybin commented

2018-03-26 14:20:45 +02:00

@mont29, this doesn't necessarily mean that both threads are running outside. What could be happening is:

Thread 0 invokes depsgraph update
Depsgraph updates relations, kicks out some object from it
Thread 14 updates scene collections, freeing old ID properties, allocating new ones.
Depsgraph update is done
Thread 0 continues to work, accesses object which was somewhere in the past evaluated, but is NOT evaluated by the depsgraph update from few lines above.
You have crash.

Either that, or object is not tagged for update, and hence it doesn't receive updated pointer for collection properties.

The point here is: it is not necessarily a concurrent access, such a bad memory access might happen in a "linear" fashion.

What we can try doing, is to NULL all runtime fields when object goes away from graph. But don't think it's really needed to be done now, with CoW this will happen auto-magically: evaluated fields will only be written to object owned by Depsgraph. So undo system will not see them. And then, if some object is not part of depsgraph anymore, it is gone forever.

@mont29, this doesn't necessarily mean that both threads are running outside. What could be happening is: - Thread 0 invokes depsgraph update - Depsgraph updates relations, kicks out some object from it - Thread 14 updates scene collections, freeing old ID properties, allocating new ones. - Depsgraph update is done - Thread 0 continues to work, accesses object which was somewhere in the past evaluated, but is NOT evaluated by the depsgraph update from few lines above. - You have crash. Either that, or object is not tagged for update, and hence it doesn't receive updated pointer for collection properties. The point here is: it is not necessarily a concurrent access, such a bad memory access might happen in a "linear" fashion. What we can try doing, is to NULL all runtime fields when object goes away from graph. But don't think it's really needed to be done now, with CoW this will happen auto-magically: evaluated fields will only be written to object owned by Depsgraph. So undo system will not see them. And then, if some object is not part of depsgraph anymore, it is gone forever.

Bastien Montagne commented

2018-03-26 15:02:07 +02:00

Yes, agree that CoW should make that kind of issue history, so let’s wait and hope for the Mighty CoW! :)

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

Bad data editing by during DEG evaluation? #53800